Jul 08 19:39:54.302462 ip-10-0-129-198 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 19:39:54.302473 ip-10-0-129-198 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 19:39:54.302480 ip-10-0-129-198 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 19:39:54.302712 ip-10-0-129-198 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 19:40:04.515627 ip-10-0-129-198 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 19:40:04.515647 ip-10-0-129-198 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot ee915c5aed2c4f32b78a59ec365f0b8d -- Jul 08 19:41:58.184335 ip-10-0-129-198 systemd[1]: Starting Kubernetes Kubelet... Jul 08 19:41:58.611362 ip-10-0-129-198 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 19:41:58.611362 ip-10-0-129-198 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 19:41:58.611362 ip-10-0-129-198 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 19:41:58.611362 ip-10-0-129-198 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 19:41:58.611362 ip-10-0-129-198 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 19:41:58.612064 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.611453 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 19:41:58.617287 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617266 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:41:58.617287 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617283 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:41:58.617287 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617287 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:41:58.617287 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617290 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617294 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617297 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617300 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617302 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617305 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617308 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617311 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617313 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617316 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617319 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617321 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617324 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617328 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617331 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617334 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617337 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617340 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617343 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617345 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:41:58.617423 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617348 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617354 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617357 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617360 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617363 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617366 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617368 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617371 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617374 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617376 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617379 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617382 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617385 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617388 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617392 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617395 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617398 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617401 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617404 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617406 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:41:58.617915 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617409 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617412 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617414 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617417 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617420 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617422 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617425 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617428 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617431 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617434 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617437 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617440 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617468 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617472 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617477 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617480 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617483 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617486 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617489 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617492 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:41:58.618479 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617495 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617498 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617501 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617504 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617506 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617509 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617513 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617517 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617519 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617522 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617525 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617528 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617531 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617533 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617536 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617539 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617542 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617545 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617548 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617551 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:41:58.619050 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617554 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:41:58.619610 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617556 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:41:58.619610 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617559 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:41:58.619610 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617562 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:41:58.619610 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617564 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:41:58.619610 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617567 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:41:58.619610 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617569 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:41:58.619610 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617572 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:41:58.619610 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617575 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:41:58.619610 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617577 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:41:58.619610 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617583 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:41:58.619610 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617588 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:41:58.619610 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617592 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:41:58.619610 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617596 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:41:58.619610 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617600 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:41:58.619610 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.617604 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:41:58.619610 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618367 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:41:58.619610 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618375 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:41:58.619610 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618379 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618382 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618385 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618388 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618392 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618395 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618398 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618402 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618405 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618408 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618411 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618418 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618421 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618424 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618427 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618430 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618433 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618436 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618439 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:41:58.620078 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618441 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618446 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618450 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618453 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618456 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618460 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618464 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618467 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618470 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618473 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618476 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618479 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618482 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618486 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618489 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618492 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618495 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618498 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618501 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618504 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:41:58.620636 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618506 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618509 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618512 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618515 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618517 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618521 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618523 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618526 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618529 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618531 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618534 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618536 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618539 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618543 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618546 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618557 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618562 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618566 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618569 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618572 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:41:58.621224 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618575 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618578 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618582 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618585 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618588 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618591 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618596 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618599 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618603 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618606 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618609 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618612 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618615 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618618 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618621 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618625 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618628 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618631 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618634 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:41:58.621738 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618637 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618639 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618642 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618644 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618647 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618650 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618652 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618655 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618658 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618661 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618664 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618667 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618669 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618672 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618675 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618677 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618680 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618683 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.618686 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619089 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:41:58.622247 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619094 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619096 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619100 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619103 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619122 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619126 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619129 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619132 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619135 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619138 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619141 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619144 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619147 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619150 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619153 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619155 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619158 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619160 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619164 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619166 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:41:58.622826 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619170 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619173 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619176 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619179 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619181 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619184 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619187 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619190 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619193 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619195 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619198 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619201 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619204 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619207 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619210 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619212 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619215 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619218 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619221 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:41:58.623442 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619224 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619227 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619230 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619235 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619238 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619242 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619245 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619248 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619251 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619254 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619256 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619259 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619262 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619266 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619268 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619271 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619274 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619276 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619279 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:41:58.623998 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619282 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619286 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619288 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619291 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619294 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619297 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619300 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619303 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619306 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619309 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619311 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619314 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619317 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619321 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619325 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619328 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619331 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619334 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619338 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619342 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:41:58.624561 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619345 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619348 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619351 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619353 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619356 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619359 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619361 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619364 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619366 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619369 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619372 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619374 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619378 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619381 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619384 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619386 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619394 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619396 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619399 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.619402 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:41:58.625180 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619479 2568 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619486 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619493 2568 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619498 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619503 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619508 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619515 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619522 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619528 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619533 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619538 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619541 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619546 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619549 2568 flags.go:64] FLAG: --cgroup-root="" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619552 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619555 2568 flags.go:64] FLAG: --client-ca-file="" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619558 2568 flags.go:64] FLAG: --cloud-provider="external" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619561 2568 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619566 2568 flags.go:64] FLAG: --cluster-domain="" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619569 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619572 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619575 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619579 2568 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619584 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 19:41:58.625761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619587 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619590 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619593 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619597 2568 flags.go:64] FLAG: --contention-profiling="false" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619602 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619605 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619609 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619612 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619617 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619620 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619624 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619627 2568 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619630 2568 flags.go:64] FLAG: --enable-server="true" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619633 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619637 2568 flags.go:64] FLAG: --event-burst="100" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619641 2568 flags.go:64] FLAG: --event-qps="50" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619644 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619647 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619650 2568 flags.go:64] FLAG: --eviction-hard="" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619654 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619657 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619661 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619664 2568 flags.go:64] FLAG: --eviction-soft="" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619667 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619670 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 19:41:58.626465 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619673 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619676 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619679 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619683 2568 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619686 2568 flags.go:64] FLAG: --feature-gates="" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619690 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619693 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619696 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619699 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619702 2568 flags.go:64] FLAG: --healthz-port="10248" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619705 2568 flags.go:64] FLAG: --help="false" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619710 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-129-198.ec2.internal" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619713 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619717 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619720 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619723 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619727 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619730 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619734 2568 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619737 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619740 2568 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619743 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619747 2568 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619750 2568 flags.go:64] FLAG: --kube-reserved="" Jul 08 19:41:58.627264 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619753 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619757 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619760 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619763 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619766 2568 flags.go:64] FLAG: --lock-file="" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619770 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619773 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619777 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619783 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619786 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619789 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619792 2568 flags.go:64] FLAG: --logging-format="text" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619795 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619798 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619802 2568 flags.go:64] FLAG: --manifest-url="" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619805 2568 flags.go:64] FLAG: --manifest-url-header="" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619809 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619813 2568 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619817 2568 flags.go:64] FLAG: --max-pods="110" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619821 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619824 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619827 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619831 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619834 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619837 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 19:41:58.627920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619842 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619850 2568 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619853 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619856 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619859 2568 flags.go:64] FLAG: --pod-cidr="" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619862 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619867 2568 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619870 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619874 2568 flags.go:64] FLAG: --pods-per-core="0" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619877 2568 flags.go:64] FLAG: --port="10250" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619880 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619884 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07d9c27004ddd63d0" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619887 2568 flags.go:64] FLAG: --qos-reserved="" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619892 2568 flags.go:64] FLAG: --read-only-port="10255" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619895 2568 flags.go:64] FLAG: --register-node="true" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619898 2568 flags.go:64] FLAG: --register-with-taints="" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619902 2568 flags.go:64] FLAG: --registry-burst="10" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619905 2568 flags.go:64] FLAG: --registry-qps="5" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619908 2568 flags.go:64] FLAG: --reserved-cpus="" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619911 2568 flags.go:64] FLAG: --reserved-memory="" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619915 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619919 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619922 2568 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619925 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619928 2568 flags.go:64] FLAG: --runonce="false" Jul 08 19:41:58.628605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619931 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619934 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619939 2568 flags.go:64] FLAG: --seccomp-default="false" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619942 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619945 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619948 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619951 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619956 2568 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619959 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619962 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619965 2568 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619968 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619971 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619974 2568 flags.go:64] FLAG: --system-cgroups="" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619977 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619983 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619986 2568 flags.go:64] FLAG: --tls-cert-file="" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619989 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619994 2568 flags.go:64] FLAG: --tls-min-version="" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.619997 2568 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.620000 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.620003 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.620006 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.620009 2568 flags.go:64] FLAG: --v="2" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.620014 2568 flags.go:64] FLAG: --version="false" Jul 08 19:41:58.629276 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.620018 2568 flags.go:64] FLAG: --vmodule="" Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.620030 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.620033 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620171 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620175 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620179 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620182 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620185 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620189 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620193 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620197 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620200 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620202 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620205 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620210 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620213 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620216 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620219 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620222 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620225 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:41:58.629932 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620228 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620230 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620233 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620236 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620239 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620242 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620244 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620247 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620250 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620271 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620274 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620277 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620279 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620282 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620285 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620288 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620291 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620293 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620296 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620299 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:41:58.630497 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620302 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620306 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620309 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620312 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620315 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620318 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620322 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620325 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620328 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620331 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620333 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620336 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620339 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620342 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620344 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620347 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620350 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620352 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620355 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620358 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:41:58.631027 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620360 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620363 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620366 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620369 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620372 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620374 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620379 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620383 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620386 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620389 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620392 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620395 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620398 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620402 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620405 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620408 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620411 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620414 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620419 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:41:58.631614 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620422 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620425 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620428 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620430 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620433 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620436 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620440 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620443 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620447 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620449 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620452 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620455 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620458 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620461 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620464 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620466 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620472 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620475 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620477 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:41:58.632106 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620480 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:41:58.632616 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620483 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:41:58.632616 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620487 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:41:58.632616 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.620491 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:41:58.632616 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.621314 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 19:41:58.632616 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.628518 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 19:41:58.632616 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.628534 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 19:41:58.632616 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.629475 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 19:41:58.632616 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.629483 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 19:41:58.632616 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629545 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:41:58.632616 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629550 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:41:58.632616 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629554 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:41:58.632616 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629558 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:41:58.632616 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629561 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:41:58.632616 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629565 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629568 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629570 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629573 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629576 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629579 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629582 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629585 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629587 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629590 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629593 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629596 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629599 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629602 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629605 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629608 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629611 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629614 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629617 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629620 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:41:58.633009 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629622 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629625 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629628 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629631 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629635 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629638 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629641 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629644 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629646 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629649 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629651 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629654 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629657 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629659 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629662 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629665 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629668 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629670 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629673 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629676 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:41:58.633552 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629678 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629681 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629684 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629686 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629689 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629692 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629694 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629697 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629701 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629703 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629706 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629709 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629713 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629716 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629719 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629722 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629725 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629728 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629731 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629734 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:41:58.634061 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629737 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629739 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629742 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629745 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629748 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629750 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629753 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629756 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629759 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629761 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629764 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629767 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629769 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629773 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629775 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629780 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629784 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629788 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629791 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:41:58.634592 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629795 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:41:58.635082 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629798 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:41:58.635082 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629802 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:41:58.635082 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629805 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:41:58.635082 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629808 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:41:58.635082 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629811 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:41:58.635082 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629813 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:41:58.635082 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629816 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:41:58.635082 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629819 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:41:58.635082 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629823 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:41:58.635082 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629826 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:41:58.635082 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629828 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:41:58.635082 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629831 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:41:58.635082 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629834 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:41:58.635082 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629836 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:41:58.635082 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.629842 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629963 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629968 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629971 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629974 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629977 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629980 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629983 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629986 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629989 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629992 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629994 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.629997 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630000 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630002 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630005 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630008 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630011 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630014 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:41:58.635516 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630018 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630022 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630025 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630028 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630031 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630034 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630038 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630041 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630044 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630047 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630050 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630053 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630056 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630060 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630062 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630065 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630068 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630070 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630073 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630076 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:41:58.636002 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630094 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630097 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630100 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630104 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630108 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630113 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630117 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630122 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630125 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630128 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630130 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630133 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630135 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630139 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630142 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630145 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630148 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630151 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630155 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:41:58.636556 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630158 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630162 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630165 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630168 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630171 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630174 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630176 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630179 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630182 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630185 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630187 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630191 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630194 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630197 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630199 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630202 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630205 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630207 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630210 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630213 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:41:58.637062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630215 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630218 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630221 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630223 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630226 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630229 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630231 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630234 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630237 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630239 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630242 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630245 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630248 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630254 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630257 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630260 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630262 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630265 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630268 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630270 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:41:58.637587 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630273 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:41:58.638105 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:58.630276 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:41:58.638105 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.630281 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 19:41:58.638105 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.630395 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 19:41:58.638105 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.632503 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 19:41:58.638105 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.634164 2568 server.go:1019] "Starting client certificate rotation" Jul 08 19:41:58.638105 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.634273 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 19:41:58.638105 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.634317 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 19:41:58.638105 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.636913 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 19:41:58.640050 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.640007 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 19:41:58.650360 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.650336 2568 log.go:25] "Validated CRI v1 runtime API" Jul 08 19:41:58.652558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.652541 2568 log.go:25] "Validated CRI v1 image API" Jul 08 19:41:58.653333 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.653319 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 19:41:58.656094 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.656059 2568 fs.go:135] Filesystem UUIDs: map[1b857cb0-c461-4d06-92af-04cac5d89ec8:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 e5fb4ca4-d1e9-4f1d-9365-7b531dfe1d86:/dev/nvme0n1p4] Jul 08 19:41:58.656152 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.656095 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 19:41:58.661600 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.661480 2568 manager.go:216] Machine: {Timestamp:2026-07-08 19:41:58.660106091 +0000 UTC m=+0.353415571 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098915 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2604f11c8a9149e7c869429ecd1ca2 SystemUUID:ec2604f1-1c8a-9149-e7c8-69429ecd1ca2 BootID:ee915c5a-ed2c-4f32-b78a-59ec365f0b8d Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f0:a4:6a:76:1b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f0:a4:6a:76:1b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:28:8f:f5:a8:af Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 19:41:58.661600 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.661594 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 19:41:58.661740 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.661677 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 19:41:58.665389 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.665355 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 19:41:58.665544 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.665394 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-198.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 19:41:58.666128 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.666098 2568 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 19:41:58.666164 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.666130 2568 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 19:41:58.666164 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.666144 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 19:41:58.666779 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.666768 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 19:41:58.667583 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.667568 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 19:41:58.668006 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.667994 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 19:41:58.671348 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.671335 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 08 19:41:58.671478 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.671468 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 19:41:58.671521 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.671499 2568 kubelet.go:486] "Attempting to sync node with API server" Jul 08 19:41:58.671521 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.671509 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 19:41:58.671588 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.671524 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 19:41:58.671588 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.671546 2568 kubelet.go:398] "Adding apiserver pod source" Jul 08 19:41:58.671588 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.671557 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 19:41:58.672061 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.672040 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 19:41:58.672185 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.672072 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 19:41:58.674496 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.674477 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 19:41:58.676431 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.676416 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 19:41:58.676498 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.676447 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 19:41:58.677575 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.677563 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 19:41:58.677618 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.677579 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 19:41:58.677618 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.677586 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 19:41:58.677618 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.677591 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 19:41:58.677618 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.677600 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 19:41:58.677618 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.677606 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 19:41:58.677618 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.677614 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 19:41:58.677618 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.677619 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 19:41:58.677813 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.677630 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 19:41:58.677813 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.677636 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 19:41:58.677813 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.677645 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 19:41:58.677813 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.677651 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 19:41:58.678595 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.678582 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 19:41:58.678646 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.678595 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 19:41:58.683367 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.683346 2568 server.go:1268] "Started kubelet" Jul 08 19:41:58.683495 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.683420 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 19:41:58.683561 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.683501 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 19:41:58.683697 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.683663 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 19:41:58.684522 ip-10-0-129-198 systemd[1]: Started Kubernetes Kubelet. Jul 08 19:41:58.684890 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:58.684861 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-198.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 19:41:58.684890 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.684883 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-198.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 19:41:58.685020 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:58.684902 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 19:41:58.685778 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.685671 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 19:41:58.688701 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.688683 2568 server.go:310] "Adding debug handlers to kubelet server" Jul 08 19:41:58.689018 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:58.688993 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 19:41:58.692516 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:58.691437 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-198.ec2.internal.18c067ecaeeac0e7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-198.ec2.internal,UID:ip-10-0-129-198.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-198.ec2.internal,},FirstTimestamp:2026-07-08 19:41:58.683279591 +0000 UTC m=+0.376589070,LastTimestamp:2026-07-08 19:41:58.683279591 +0000 UTC m=+0.376589070,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-198.ec2.internal,}" Jul 08 19:41:58.693840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.693816 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 19:41:58.693946 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.693824 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 19:41:58.693946 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:58.693923 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-198.ec2.internal\" not found" Jul 08 19:41:58.694508 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.694485 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 19:41:58.694508 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.694511 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 19:41:58.694648 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.694602 2568 reconstruct.go:97] "Volume reconstruction finished" Jul 08 19:41:58.694648 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.694614 2568 reconciler.go:29] "Reconciler: start to sync state" Jul 08 19:41:58.694770 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.694747 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 19:41:58.695096 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.695058 2568 factory.go:153] Registering CRI-O factory Jul 08 19:41:58.695096 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.695099 2568 factory.go:223] Registration of the crio container factory successfully Jul 08 19:41:58.695267 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.695147 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 19:41:58.695267 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.695157 2568 factory.go:55] Registering systemd factory Jul 08 19:41:58.695267 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.695166 2568 factory.go:223] Registration of the systemd container factory successfully Jul 08 19:41:58.695267 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.695196 2568 factory.go:103] Registering Raw factory Jul 08 19:41:58.695267 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.695209 2568 manager.go:1196] Started watching for new ooms in manager Jul 08 19:41:58.695791 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.695620 2568 manager.go:349] Starting recovery of all containers Jul 08 19:41:58.695791 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.695773 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w4cw5" Jul 08 19:41:58.704053 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.704028 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w4cw5" Jul 08 19:41:58.705209 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.705179 2568 manager.go:354] Recovery completed Jul 08 19:41:58.706424 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:58.706385 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 19:41:58.706644 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:58.706617 2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-198.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 19:41:58.711745 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.711728 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:41:58.713717 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.713698 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:41:58.713787 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.713735 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:41:58.713787 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.713749 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:41:58.714391 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.714374 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 19:41:58.714391 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.714387 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:41:58.714391 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.714392 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 19:41:58.714590 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.714411 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 08 19:41:58.715135 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.715101 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:41:58.715209 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.715152 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:41:58.715209 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.715170 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:41:58.715772 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.715756 2568 policy_none.go:49] "None policy: Start" Jul 08 19:41:58.715772 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.715774 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 19:41:58.715899 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.715788 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 19:41:58.716127 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:58.715999 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-198.ec2.internal.18c067ecb0bb2d5d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-198.ec2.internal,UID:ip-10-0-129-198.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-198.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-198.ec2.internal,},FirstTimestamp:2026-07-08 19:41:58.713716061 +0000 UTC m=+0.407025544,LastTimestamp:2026-07-08 19:41:58.713716061 +0000 UTC m=+0.407025544,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-198.ec2.internal,}" Jul 08 19:41:58.716879 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.716866 2568 policy_none.go:47] "Start" Jul 08 19:41:58.753983 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.753750 2568 manager.go:337] "Starting Device Plugin manager" Jul 08 19:41:58.753983 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:58.753965 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 19:41:58.753983 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.753981 2568 server.go:87] "Starting device plugin registration server" Jul 08 19:41:58.754318 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.754303 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 19:41:58.754380 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.754324 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 19:41:58.754450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.754432 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 19:41:58.754537 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.754525 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 19:41:58.754590 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.754539 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 19:41:58.755098 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:58.755080 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 19:41:58.755183 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:58.755134 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-198.ec2.internal\" not found" Jul 08 19:41:58.824062 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.824018 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 19:41:58.825417 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.825380 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 19:41:58.825498 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.825428 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 19:41:58.825498 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.825458 2568 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 19:41:58.825596 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:58.825502 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 19:41:58.828218 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.828198 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:41:58.854492 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.854471 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:41:58.855511 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.855491 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:41:58.855582 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.855529 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:41:58.855582 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.855544 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:41:58.855582 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.855569 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-198.ec2.internal" Jul 08 19:41:58.864188 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.864172 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-198.ec2.internal" Jul 08 19:41:58.864241 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:58.864197 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-198.ec2.internal\": node \"ip-10-0-129-198.ec2.internal\" not found" Jul 08 19:41:58.884833 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:58.884809 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-198.ec2.internal\" not found" Jul 08 19:41:58.926019 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.925985 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-198.ec2.internal"] Jul 08 19:41:58.926122 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.926089 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:41:58.927163 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.927147 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:41:58.927242 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.927179 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:41:58.927242 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.927190 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:41:58.928576 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.928562 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:41:58.928708 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.928693 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal" Jul 08 19:41:58.928748 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.928726 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:41:58.929374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.929359 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:41:58.929450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.929381 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:41:58.929450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.929391 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:41:58.929450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.929428 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:41:58.929450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.929449 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:41:58.929575 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.929459 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:41:58.930587 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.930573 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-198.ec2.internal" Jul 08 19:41:58.930636 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.930598 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:41:58.931353 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.931332 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:41:58.931454 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.931359 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:41:58.931454 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.931372 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:41:58.948728 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:58.948707 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-198.ec2.internal\" not found" node="ip-10-0-129-198.ec2.internal" Jul 08 19:41:58.952937 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:58.952916 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-198.ec2.internal\" not found" node="ip-10-0-129-198.ec2.internal" Jul 08 19:41:58.985771 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:58.985739 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-198.ec2.internal\" not found" Jul 08 19:41:58.996602 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.996579 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ae59f7384267d6a13c3e95812346171-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal\" (UID: \"3ae59f7384267d6a13c3e95812346171\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal" Jul 08 19:41:58.996707 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.996607 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ae59f7384267d6a13c3e95812346171-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal\" (UID: \"3ae59f7384267d6a13c3e95812346171\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal" Jul 08 19:41:58.996707 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:58.996625 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4518754363eeef80e023637afe5321de-config\") pod \"kube-apiserver-proxy-ip-10-0-129-198.ec2.internal\" (UID: \"4518754363eeef80e023637afe5321de\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-198.ec2.internal" Jul 08 19:41:59.086825 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:59.086794 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-198.ec2.internal\" not found" Jul 08 19:41:59.096874 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:59.096844 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ae59f7384267d6a13c3e95812346171-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal\" (UID: \"3ae59f7384267d6a13c3e95812346171\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal" Jul 08 19:41:59.097030 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:59.096888 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ae59f7384267d6a13c3e95812346171-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal\" (UID: \"3ae59f7384267d6a13c3e95812346171\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal" Jul 08 19:41:59.097030 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:59.096912 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4518754363eeef80e023637afe5321de-config\") pod \"kube-apiserver-proxy-ip-10-0-129-198.ec2.internal\" (UID: \"4518754363eeef80e023637afe5321de\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-198.ec2.internal" Jul 08 19:41:59.097030 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:59.096967 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4518754363eeef80e023637afe5321de-config\") pod \"kube-apiserver-proxy-ip-10-0-129-198.ec2.internal\" (UID: \"4518754363eeef80e023637afe5321de\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-198.ec2.internal" Jul 08 19:41:59.097030 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:59.096968 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ae59f7384267d6a13c3e95812346171-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal\" (UID: \"3ae59f7384267d6a13c3e95812346171\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal" Jul 08 19:41:59.097030 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:59.096997 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ae59f7384267d6a13c3e95812346171-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal\" (UID: \"3ae59f7384267d6a13c3e95812346171\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal" Jul 08 19:41:59.187133 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:59.187098 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-198.ec2.internal\" not found" Jul 08 19:41:59.255796 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:59.255743 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal" Jul 08 19:41:59.256955 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:59.256938 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-198.ec2.internal" Jul 08 19:41:59.287269 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:59.287239 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-198.ec2.internal\" not found" Jul 08 19:41:59.393288 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:59.393190 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-198.ec2.internal\" not found" Jul 08 19:41:59.498913 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:59.498871 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-198.ec2.internal\" not found" Jul 08 19:41:59.599188 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:59.599153 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-198.ec2.internal\" not found" Jul 08 19:41:59.635707 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:59.635673 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 19:41:59.636328 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:59.635847 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 19:41:59.700052 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:59.699970 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-198.ec2.internal\" not found" Jul 08 19:41:59.707986 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:59.707950 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 19:36:59 +0000 UTC" deadline="2028-02-05 23:10:49.533742195 +0000 UTC" Jul 08 19:41:59.707986 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:59.707982 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13851h28m49.825763898s" Jul 08 19:41:59.724233 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:59.724208 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:41:59.800471 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:59.800437 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-198.ec2.internal\" not found" Jul 08 19:41:59.844096 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:59.844066 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4518754363eeef80e023637afe5321de.slice/crio-d8c6ade1cf6b958bfc7ecb15330e19772eb4aba21e2d0f6184a2b9ddbcf36a55 WatchSource:0}: Error finding container d8c6ade1cf6b958bfc7ecb15330e19772eb4aba21e2d0f6184a2b9ddbcf36a55: Status 404 returned error can't find the container with id d8c6ade1cf6b958bfc7ecb15330e19772eb4aba21e2d0f6184a2b9ddbcf36a55 Jul 08 19:41:59.844573 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:41:59.844543 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ae59f7384267d6a13c3e95812346171.slice/crio-826d1ca095b3bcdf4ea876c006e8fcc77a5a5729c814293c7f913093da165f85 WatchSource:0}: Error finding container 826d1ca095b3bcdf4ea876c006e8fcc77a5a5729c814293c7f913093da165f85: Status 404 returned error can't find the container with id 826d1ca095b3bcdf4ea876c006e8fcc77a5a5729c814293c7f913093da165f85 Jul 08 19:41:59.851372 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:41:59.851350 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:41:59.900871 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:41:59.900838 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-198.ec2.internal\" not found" Jul 08 19:42:00.001796 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:00.001714 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-198.ec2.internal\" not found" Jul 08 19:42:00.004641 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.004626 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:42:00.099231 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.099191 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal" Jul 08 19:42:00.110572 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.110547 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 19:42:00.112747 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.112732 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-198.ec2.internal" Jul 08 19:42:00.121700 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.121681 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 19:42:00.266388 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.266298 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:42:00.663653 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.663621 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:42:00.674580 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.674542 2568 apiserver.go:52] "Watching apiserver" Jul 08 19:42:00.684666 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.684629 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 19:42:00.685157 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.685120 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-62sgn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal","openshift-multus/multus-additional-cni-plugins-g5gnj","openshift-network-operator/iptables-alerter-xhjd9","openshift-ovn-kubernetes/ovnkube-node-lsfpc","kube-system/global-pull-secret-syncer-82lzg","kube-system/konnectivity-agent-279bc","kube-system/kube-apiserver-proxy-ip-10-0-129-198.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv","openshift-cluster-node-tuning-operator/tuned-m7wzx","openshift-multus/multus-h77hk","openshift-multus/network-metrics-daemon-s7hbd","openshift-network-diagnostics/network-check-target-q6qvd","openshift-dns/node-resolver-rz46d"] Jul 08 19:42:00.686822 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.686793 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-279bc" Jul 08 19:42:00.688069 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.688022 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:00.688175 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:00.688116 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6qvd" podUID="dc75384f-fbaa-4ddf-b258-02a4024686aa" Jul 08 19:42:00.690558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.689963 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 19:42:00.690558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.690081 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.690558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.690166 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 19:42:00.690558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.690210 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-grcl8\"" Jul 08 19:42:00.691331 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.691309 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xhjd9" Jul 08 19:42:00.692675 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.692654 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.693224 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.693204 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 19:42:00.693592 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.693570 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 19:42:00.693592 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.693589 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-v42gv\"" Jul 08 19:42:00.693725 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.693603 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 19:42:00.693725 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.693634 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 19:42:00.693918 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.693896 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 19:42:00.693918 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.693913 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:00.694032 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:00.693974 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82lzg" podUID="3184c318-9df8-4f30-92e9-5bf8786149e9" Jul 08 19:42:00.694632 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.694613 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:42:00.694821 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.694805 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 19:42:00.695041 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.695025 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rg4h2\"" Jul 08 19:42:00.695238 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.695221 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 19:42:00.695542 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.695523 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 19:42:00.695721 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.695704 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bjl5c\"" Jul 08 19:42:00.696595 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.696163 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 19:42:00.699457 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.699245 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 19:42:00.699644 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.699622 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 19:42:00.699738 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.699711 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-62sgn" Jul 08 19:42:00.700076 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.700054 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 19:42:00.700224 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.700144 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 19:42:00.700631 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.700581 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 19:42:00.701398 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.701378 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rz46d" Jul 08 19:42:00.701734 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.701712 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.703523 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.703434 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.703692 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.703662 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 19:42:00.704115 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.704094 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 19:42:00.704582 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.704542 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kdg82\"" Jul 08 19:42:00.704727 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.704696 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 19:42:00.704801 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.704724 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 19:42:00.704801 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.704785 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 19:42:00.704946 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.704922 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 19:42:00.705523 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705497 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/33ffe4eb-0ee2-4e34-a03b-b1f06bb98602-konnectivity-ca\") pod \"konnectivity-agent-279bc\" (UID: \"33ffe4eb-0ee2-4e34-a03b-b1f06bb98602\") " pod="kube-system/konnectivity-agent-279bc" Jul 08 19:42:00.705615 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705535 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-etc-openvswitch\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.705615 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705565 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-run-ovn-kubernetes\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.705615 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705592 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret\") pod \"global-pull-secret-syncer-82lzg\" (UID: \"3184c318-9df8-4f30-92e9-5bf8786149e9\") " pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:00.705785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705618 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-registration-dir\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.705785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705647 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-sys-fs\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.705785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705673 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt86s\" (UniqueName: \"kubernetes.io/projected/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-kube-api-access-nt86s\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.705785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705696 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d277414e-55a2-411f-852d-aae01918e1c4-os-release\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.705785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705723 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d277414e-55a2-411f-852d-aae01918e1c4-cni-binary-copy\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.705785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705750 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d277414e-55a2-411f-852d-aae01918e1c4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.705785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705778 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f6d7662e-e8aa-4e30-ad77-7956f718b804-iptables-alerter-script\") pod \"iptables-alerter-xhjd9\" (UID: \"f6d7662e-e8aa-4e30-ad77-7956f718b804\") " pod="openshift-network-operator/iptables-alerter-xhjd9" Jul 08 19:42:00.706155 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705802 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsqdd\" (UniqueName: \"kubernetes.io/projected/f6d7662e-e8aa-4e30-ad77-7956f718b804-kube-api-access-dsqdd\") pod \"iptables-alerter-xhjd9\" (UID: \"f6d7662e-e8aa-4e30-ad77-7956f718b804\") " pod="openshift-network-operator/iptables-alerter-xhjd9" Jul 08 19:42:00.706155 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705835 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-slash\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.706155 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705861 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-run-ovn\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.706155 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705885 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-cni-netd\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.706155 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705909 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/33ffe4eb-0ee2-4e34-a03b-b1f06bb98602-agent-certs\") pod \"konnectivity-agent-279bc\" (UID: \"33ffe4eb-0ee2-4e34-a03b-b1f06bb98602\") " pod="kube-system/konnectivity-agent-279bc" Jul 08 19:42:00.706155 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705934 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-run-netns\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.706155 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705959 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-var-lib-openvswitch\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.706155 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.705983 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-log-socket\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.706155 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706006 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45d42bef-ca8b-42c5-8b2a-90485f01ce14-env-overrides\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.706155 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706032 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45d42bef-ca8b-42c5-8b2a-90485f01ce14-ovnkube-script-lib\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.706155 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706056 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.706155 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706080 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a5cf071e-9c63-46a7-8fef-f1f11d2a43cd-serviceca\") pod \"node-ca-62sgn\" (UID: \"a5cf071e-9c63-46a7-8fef-f1f11d2a43cd\") " pod="openshift-image-registry/node-ca-62sgn" Jul 08 19:42:00.706155 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706123 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wppnk\" (UniqueName: \"kubernetes.io/projected/a5cf071e-9c63-46a7-8fef-f1f11d2a43cd-kube-api-access-wppnk\") pod \"node-ca-62sgn\" (UID: \"a5cf071e-9c63-46a7-8fef-f1f11d2a43cd\") " pod="openshift-image-registry/node-ca-62sgn" Jul 08 19:42:00.706155 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706152 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-run-systemd\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.706837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706178 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-cni-bin\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.706837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706202 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5cf071e-9c63-46a7-8fef-f1f11d2a43cd-host\") pod \"node-ca-62sgn\" (UID: \"a5cf071e-9c63-46a7-8fef-f1f11d2a43cd\") " pod="openshift-image-registry/node-ca-62sgn" Jul 08 19:42:00.706837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706241 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d277414e-55a2-411f-852d-aae01918e1c4-cnibin\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.706837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706267 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-systemd-units\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.706837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706292 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qr5z\" (UniqueName: \"kubernetes.io/projected/45d42bef-ca8b-42c5-8b2a-90485f01ce14-kube-api-access-8qr5z\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.706837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706318 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3184c318-9df8-4f30-92e9-5bf8786149e9-kubelet-config\") pod \"global-pull-secret-syncer-82lzg\" (UID: \"3184c318-9df8-4f30-92e9-5bf8786149e9\") " pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:00.706837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706342 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3184c318-9df8-4f30-92e9-5bf8786149e9-dbus\") pod \"global-pull-secret-syncer-82lzg\" (UID: \"3184c318-9df8-4f30-92e9-5bf8786149e9\") " pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:00.706837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706374 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-socket-dir\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.706837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706399 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d277414e-55a2-411f-852d-aae01918e1c4-system-cni-dir\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.706837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706443 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7x4d\" (UniqueName: \"kubernetes.io/projected/d277414e-55a2-411f-852d-aae01918e1c4-kube-api-access-w7x4d\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.706837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706470 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-run-openvswitch\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.706837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706501 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-node-log\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.706837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706542 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45d42bef-ca8b-42c5-8b2a-90485f01ce14-ovnkube-config\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.706837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706571 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89clm\" (UniqueName: \"kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm\") pod \"network-check-target-q6qvd\" (UID: \"dc75384f-fbaa-4ddf-b258-02a4024686aa\") " pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:00.706837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706601 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d277414e-55a2-411f-852d-aae01918e1c4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.706837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706627 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d277414e-55a2-411f-852d-aae01918e1c4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.707527 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706651 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-kubelet\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.707527 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706678 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.707527 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706708 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45d42bef-ca8b-42c5-8b2a-90485f01ce14-ovn-node-metrics-cert\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.707527 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706732 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-device-dir\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.707527 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706755 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-etc-selinux\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.707527 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.706778 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f6d7662e-e8aa-4e30-ad77-7956f718b804-host-slash\") pod \"iptables-alerter-xhjd9\" (UID: \"f6d7662e-e8aa-4e30-ad77-7956f718b804\") " pod="openshift-network-operator/iptables-alerter-xhjd9" Jul 08 19:42:00.708075 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.708053 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 19:42:00.708302 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.708193 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dxlr6\"" Jul 08 19:42:00.708406 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.708338 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 19:42:00.708406 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.708384 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-b2pmd\"" Jul 08 19:42:00.708815 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.708790 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.709536 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.709373 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:42:00.709536 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.709412 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 19:42:00.709536 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.709422 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-czlt8\"" Jul 08 19:42:00.711944 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.711765 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:00.711944 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.711782 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 19:42:00.711944 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:00.711832 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:42:00.712192 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.712165 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-p6brb\"" Jul 08 19:42:00.714647 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.714112 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 19:42:00.745528 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.745498 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-phm8n" Jul 08 19:42:00.755697 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.755671 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-phm8n" Jul 08 19:42:00.796159 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.796126 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 19:42:00.807584 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807540 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-etc-selinux\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.807584 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807580 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f6d7662e-e8aa-4e30-ad77-7956f718b804-host-slash\") pod \"iptables-alerter-xhjd9\" (UID: \"f6d7662e-e8aa-4e30-ad77-7956f718b804\") " pod="openshift-network-operator/iptables-alerter-xhjd9" Jul 08 19:42:00.807785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807611 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-multus-socket-dir-parent\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.807785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807635 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-host-var-lib-cni-bin\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.807785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807668 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-host-run-multus-certs\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.807785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807690 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d277414e-55a2-411f-852d-aae01918e1c4-os-release\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.807785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807717 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d277414e-55a2-411f-852d-aae01918e1c4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.807785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807733 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-etc-selinux\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.807785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807745 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-run-ovn\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.807785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807777 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwlcg\" (UniqueName: \"kubernetes.io/projected/baeda2ff-0e38-4c4a-a057-dcd4e9020371-kube-api-access-pwlcg\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.808101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807802 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-slash\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.808101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807816 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f6d7662e-e8aa-4e30-ad77-7956f718b804-host-slash\") pod \"iptables-alerter-xhjd9\" (UID: \"f6d7662e-e8aa-4e30-ad77-7956f718b804\") " pod="openshift-network-operator/iptables-alerter-xhjd9" Jul 08 19:42:00.808101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807826 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/394cd92a-488b-4386-9c3c-5e607c321956-tmp\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.808101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807850 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-system-cni-dir\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.808101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807875 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d69db7df-9f0a-4339-99b0-1d7dffa3e30e-tmp-dir\") pod \"node-resolver-rz46d\" (UID: \"d69db7df-9f0a-4339-99b0-1d7dffa3e30e\") " pod="openshift-dns/node-resolver-rz46d" Jul 08 19:42:00.808101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807899 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/33ffe4eb-0ee2-4e34-a03b-b1f06bb98602-agent-certs\") pod \"konnectivity-agent-279bc\" (UID: \"33ffe4eb-0ee2-4e34-a03b-b1f06bb98602\") " pod="kube-system/konnectivity-agent-279bc" Jul 08 19:42:00.808101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807925 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d277414e-55a2-411f-852d-aae01918e1c4-os-release\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.808101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807933 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-var-lib-openvswitch\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.808101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807959 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45d42bef-ca8b-42c5-8b2a-90485f01ce14-env-overrides\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.808101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.807986 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45d42bef-ca8b-42c5-8b2a-90485f01ce14-ovnkube-script-lib\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.808101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.808009 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-etc-sysctl-d\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.808101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.808015 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-slash\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.808101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.808043 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-run\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.808101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.808080 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-run-ovn\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.808669 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.808559 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45d42bef-ca8b-42c5-8b2a-90485f01ce14-env-overrides\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.808730 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.808701 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d277414e-55a2-411f-852d-aae01918e1c4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.808788 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.808711 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45d42bef-ca8b-42c5-8b2a-90485f01ce14-ovnkube-script-lib\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.808788 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.808754 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-var-lib-openvswitch\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.808788 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.808777 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-cnibin\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.808926 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.808810 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/baeda2ff-0e38-4c4a-a057-dcd4e9020371-multus-daemon-config\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.808926 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.808845 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.808926 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.808896 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 19:42:00.808926 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.808909 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.809153 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.808900 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a5cf071e-9c63-46a7-8fef-f1f11d2a43cd-serviceca\") pod \"node-ca-62sgn\" (UID: \"a5cf071e-9c63-46a7-8fef-f1f11d2a43cd\") " pod="openshift-image-registry/node-ca-62sgn" Jul 08 19:42:00.809153 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.808953 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wppnk\" (UniqueName: \"kubernetes.io/projected/a5cf071e-9c63-46a7-8fef-f1f11d2a43cd-kube-api-access-wppnk\") pod \"node-ca-62sgn\" (UID: \"a5cf071e-9c63-46a7-8fef-f1f11d2a43cd\") " pod="openshift-image-registry/node-ca-62sgn" Jul 08 19:42:00.809153 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.808978 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-run-systemd\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.809153 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.808994 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-cni-bin\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.809153 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809010 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-host-run-netns\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.809153 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809033 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-etc-kubernetes\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.809153 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809059 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfdsk\" (UniqueName: \"kubernetes.io/projected/3b5311a0-d5fa-4994-9071-d861d2a53b09-kube-api-access-jfdsk\") pod \"network-metrics-daemon-s7hbd\" (UID: \"3b5311a0-d5fa-4994-9071-d861d2a53b09\") " pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:00.809153 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809082 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5cf071e-9c63-46a7-8fef-f1f11d2a43cd-host\") pod \"node-ca-62sgn\" (UID: \"a5cf071e-9c63-46a7-8fef-f1f11d2a43cd\") " pod="openshift-image-registry/node-ca-62sgn" Jul 08 19:42:00.809153 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809101 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3184c318-9df8-4f30-92e9-5bf8786149e9-kubelet-config\") pod \"global-pull-secret-syncer-82lzg\" (UID: \"3184c318-9df8-4f30-92e9-5bf8786149e9\") " pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:00.809153 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809137 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3184c318-9df8-4f30-92e9-5bf8786149e9-dbus\") pod \"global-pull-secret-syncer-82lzg\" (UID: \"3184c318-9df8-4f30-92e9-5bf8786149e9\") " pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:00.809614 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809158 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-host-var-lib-cni-multus\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.809614 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809177 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxkzq\" (UniqueName: \"kubernetes.io/projected/d69db7df-9f0a-4339-99b0-1d7dffa3e30e-kube-api-access-dxkzq\") pod \"node-resolver-rz46d\" (UID: \"d69db7df-9f0a-4339-99b0-1d7dffa3e30e\") " pod="openshift-dns/node-resolver-rz46d" Jul 08 19:42:00.809614 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809202 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-etc-kubernetes\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.809614 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809233 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7x4d\" (UniqueName: \"kubernetes.io/projected/d277414e-55a2-411f-852d-aae01918e1c4-kube-api-access-w7x4d\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.809614 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809266 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-run-openvswitch\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.809614 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809282 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-etc-modprobe-d\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.809614 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809304 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r7cv\" (UniqueName: \"kubernetes.io/projected/394cd92a-488b-4386-9c3c-5e607c321956-kube-api-access-5r7cv\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.809614 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809330 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-host-var-lib-kubelet\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.809614 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809358 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d277414e-55a2-411f-852d-aae01918e1c4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.809614 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809423 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.809614 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809458 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45d42bef-ca8b-42c5-8b2a-90485f01ce14-ovn-node-metrics-cert\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.809614 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809487 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-multus-conf-dir\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.809614 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809531 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d69db7df-9f0a-4339-99b0-1d7dffa3e30e-hosts-file\") pod \"node-resolver-rz46d\" (UID: \"d69db7df-9f0a-4339-99b0-1d7dffa3e30e\") " pod="openshift-dns/node-resolver-rz46d" Jul 08 19:42:00.809614 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809559 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/33ffe4eb-0ee2-4e34-a03b-b1f06bb98602-konnectivity-ca\") pod \"konnectivity-agent-279bc\" (UID: \"33ffe4eb-0ee2-4e34-a03b-b1f06bb98602\") " pod="kube-system/konnectivity-agent-279bc" Jul 08 19:42:00.809614 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809585 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-etc-openvswitch\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.809614 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809614 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-run-ovn-kubernetes\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.810450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809633 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-run-systemd\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.810450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809641 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret\") pod \"global-pull-secret-syncer-82lzg\" (UID: \"3184c318-9df8-4f30-92e9-5bf8786149e9\") " pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:00.810450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809678 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-registration-dir\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.810450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809706 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-sys-fs\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.810450 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:00.809730 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:42:00.810450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809744 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt86s\" (UniqueName: \"kubernetes.io/projected/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-kube-api-access-nt86s\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.810450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809768 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d277414e-55a2-411f-852d-aae01918e1c4-cni-binary-copy\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.810450 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:00.809815 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret podName:3184c318-9df8-4f30-92e9-5bf8786149e9 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:01.309775156 +0000 UTC m=+3.003084653 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret") pod "global-pull-secret-syncer-82lzg" (UID: "3184c318-9df8-4f30-92e9-5bf8786149e9") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:42:00.810450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809846 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-run-openvswitch\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.810450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809875 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-cni-bin\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.810450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809950 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-run-ovn-kubernetes\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.810450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810002 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3184c318-9df8-4f30-92e9-5bf8786149e9-dbus\") pod \"global-pull-secret-syncer-82lzg\" (UID: \"3184c318-9df8-4f30-92e9-5bf8786149e9\") " pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:00.810450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810013 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5cf071e-9c63-46a7-8fef-f1f11d2a43cd-host\") pod \"node-ca-62sgn\" (UID: \"a5cf071e-9c63-46a7-8fef-f1f11d2a43cd\") " pod="openshift-image-registry/node-ca-62sgn" Jul 08 19:42:00.810450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810060 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.810450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810209 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f6d7662e-e8aa-4e30-ad77-7956f718b804-iptables-alerter-script\") pod \"iptables-alerter-xhjd9\" (UID: \"f6d7662e-e8aa-4e30-ad77-7956f718b804\") " pod="openshift-network-operator/iptables-alerter-xhjd9" Jul 08 19:42:00.810450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810243 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsqdd\" (UniqueName: \"kubernetes.io/projected/f6d7662e-e8aa-4e30-ad77-7956f718b804-kube-api-access-dsqdd\") pod \"iptables-alerter-xhjd9\" (UID: \"f6d7662e-e8aa-4e30-ad77-7956f718b804\") " pod="openshift-network-operator/iptables-alerter-xhjd9" Jul 08 19:42:00.810450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.809328 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a5cf071e-9c63-46a7-8fef-f1f11d2a43cd-serviceca\") pod \"node-ca-62sgn\" (UID: \"a5cf071e-9c63-46a7-8fef-f1f11d2a43cd\") " pod="openshift-image-registry/node-ca-62sgn" Jul 08 19:42:00.811374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810307 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-etc-openvswitch\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.811374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810345 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d277414e-55a2-411f-852d-aae01918e1c4-cni-binary-copy\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.811374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810360 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-sys-fs\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.811374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810392 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-etc-systemd\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.811374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810413 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3184c318-9df8-4f30-92e9-5bf8786149e9-kubelet-config\") pod \"global-pull-secret-syncer-82lzg\" (UID: \"3184c318-9df8-4f30-92e9-5bf8786149e9\") " pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:00.811374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810420 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-cni-netd\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.811374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810450 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-run-netns\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.811374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810469 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d277414e-55a2-411f-852d-aae01918e1c4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.811374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810476 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-log-socket\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.811374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810509 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/33ffe4eb-0ee2-4e34-a03b-b1f06bb98602-konnectivity-ca\") pod \"konnectivity-agent-279bc\" (UID: \"33ffe4eb-0ee2-4e34-a03b-b1f06bb98602\") " pod="kube-system/konnectivity-agent-279bc" Jul 08 19:42:00.811374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810533 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-log-socket\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.811374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810562 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-etc-sysconfig\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.811374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810571 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-cni-netd\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.811374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810591 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-lib-modules\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.811374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810622 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-run-netns\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.811374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810656 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/baeda2ff-0e38-4c4a-a057-dcd4e9020371-cni-binary-copy\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.811374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810682 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-host\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.812540 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810709 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/394cd92a-488b-4386-9c3c-5e607c321956-etc-tuned\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.812540 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810736 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-hostroot\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.812540 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810780 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs\") pod \"network-metrics-daemon-s7hbd\" (UID: \"3b5311a0-d5fa-4994-9071-d861d2a53b09\") " pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:00.812540 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810793 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f6d7662e-e8aa-4e30-ad77-7956f718b804-iptables-alerter-script\") pod \"iptables-alerter-xhjd9\" (UID: \"f6d7662e-e8aa-4e30-ad77-7956f718b804\") " pod="openshift-network-operator/iptables-alerter-xhjd9" Jul 08 19:42:00.812540 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810816 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d277414e-55a2-411f-852d-aae01918e1c4-cnibin\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.812540 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810848 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-systemd-units\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.812540 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810891 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d277414e-55a2-411f-852d-aae01918e1c4-cnibin\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.812540 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810905 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qr5z\" (UniqueName: \"kubernetes.io/projected/45d42bef-ca8b-42c5-8b2a-90485f01ce14-kube-api-access-8qr5z\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.812540 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810932 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-systemd-units\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.812540 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.810992 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-etc-sysctl-conf\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.812540 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811027 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-var-lib-kubelet\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.812540 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811077 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-socket-dir\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.812540 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811115 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d277414e-55a2-411f-852d-aae01918e1c4-system-cni-dir\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.812540 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811181 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-node-log\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.812540 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811208 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45d42bef-ca8b-42c5-8b2a-90485f01ce14-ovnkube-config\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.812540 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811235 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-socket-dir\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.812540 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811267 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d277414e-55a2-411f-852d-aae01918e1c4-system-cni-dir\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.813469 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811290 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-os-release\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.813469 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811306 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-node-log\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.813469 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811341 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-host-run-k8s-cni-cncf-io\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.813469 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811388 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89clm\" (UniqueName: \"kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm\") pod \"network-check-target-q6qvd\" (UID: \"dc75384f-fbaa-4ddf-b258-02a4024686aa\") " pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:00.813469 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811446 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-registration-dir\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.813469 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811495 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d277414e-55a2-411f-852d-aae01918e1c4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.813469 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811550 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-kubelet\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.813469 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811591 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-sys\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.813469 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811635 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45d42bef-ca8b-42c5-8b2a-90485f01ce14-host-kubelet\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.813469 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811679 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-multus-cni-dir\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.813469 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811689 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45d42bef-ca8b-42c5-8b2a-90485f01ce14-ovnkube-config\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.813469 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811705 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-device-dir\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.813469 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811748 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-device-dir\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.813469 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.811846 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d277414e-55a2-411f-852d-aae01918e1c4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.813469 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.813474 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/33ffe4eb-0ee2-4e34-a03b-b1f06bb98602-agent-certs\") pod \"konnectivity-agent-279bc\" (UID: \"33ffe4eb-0ee2-4e34-a03b-b1f06bb98602\") " pod="kube-system/konnectivity-agent-279bc" Jul 08 19:42:00.814395 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.814371 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45d42bef-ca8b-42c5-8b2a-90485f01ce14-ovn-node-metrics-cert\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.821007 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:00.820556 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:42:00.821007 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:00.820580 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:42:00.821007 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:00.820593 2568 projected.go:196] Error preparing data for projected volume kube-api-access-89clm for pod openshift-network-diagnostics/network-check-target-q6qvd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:42:00.821007 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:00.820653 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm podName:dc75384f-fbaa-4ddf-b258-02a4024686aa nodeName:}" failed. No retries permitted until 2026-07-08 19:42:01.320637404 +0000 UTC m=+3.013946873 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-89clm" (UniqueName: "kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm") pod "network-check-target-q6qvd" (UID: "dc75384f-fbaa-4ddf-b258-02a4024686aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:42:00.823131 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.823095 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wppnk\" (UniqueName: \"kubernetes.io/projected/a5cf071e-9c63-46a7-8fef-f1f11d2a43cd-kube-api-access-wppnk\") pod \"node-ca-62sgn\" (UID: \"a5cf071e-9c63-46a7-8fef-f1f11d2a43cd\") " pod="openshift-image-registry/node-ca-62sgn" Jul 08 19:42:00.823527 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.823500 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7x4d\" (UniqueName: \"kubernetes.io/projected/d277414e-55a2-411f-852d-aae01918e1c4-kube-api-access-w7x4d\") pod \"multus-additional-cni-plugins-g5gnj\" (UID: \"d277414e-55a2-411f-852d-aae01918e1c4\") " pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:00.824050 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.824026 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsqdd\" (UniqueName: \"kubernetes.io/projected/f6d7662e-e8aa-4e30-ad77-7956f718b804-kube-api-access-dsqdd\") pod \"iptables-alerter-xhjd9\" (UID: \"f6d7662e-e8aa-4e30-ad77-7956f718b804\") " pod="openshift-network-operator/iptables-alerter-xhjd9" Jul 08 19:42:00.825280 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.825257 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt86s\" (UniqueName: \"kubernetes.io/projected/ccc6206f-2d22-4d59-b298-4fa3d7aec4da-kube-api-access-nt86s\") pod \"aws-ebs-csi-driver-node-phpxv\" (UID: \"ccc6206f-2d22-4d59-b298-4fa3d7aec4da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:00.825379 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.825286 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qr5z\" (UniqueName: \"kubernetes.io/projected/45d42bef-ca8b-42c5-8b2a-90485f01ce14-kube-api-access-8qr5z\") pod \"ovnkube-node-lsfpc\" (UID: \"45d42bef-ca8b-42c5-8b2a-90485f01ce14\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:00.830422 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.830372 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-198.ec2.internal" event={"ID":"4518754363eeef80e023637afe5321de","Type":"ContainerStarted","Data":"d8c6ade1cf6b958bfc7ecb15330e19772eb4aba21e2d0f6184a2b9ddbcf36a55"} Jul 08 19:42:00.831616 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.831584 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal" event={"ID":"3ae59f7384267d6a13c3e95812346171","Type":"ContainerStarted","Data":"826d1ca095b3bcdf4ea876c006e8fcc77a5a5729c814293c7f913093da165f85"} Jul 08 19:42:00.912228 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912187 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-host-run-netns\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.912428 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912264 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-etc-kubernetes\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.912428 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912185 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-host-run-netns\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.912428 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912293 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfdsk\" (UniqueName: \"kubernetes.io/projected/3b5311a0-d5fa-4994-9071-d861d2a53b09-kube-api-access-jfdsk\") pod \"network-metrics-daemon-s7hbd\" (UID: \"3b5311a0-d5fa-4994-9071-d861d2a53b09\") " pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:00.912428 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912346 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-host-var-lib-cni-multus\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.912428 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912366 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-etc-kubernetes\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.912428 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912376 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxkzq\" (UniqueName: \"kubernetes.io/projected/d69db7df-9f0a-4339-99b0-1d7dffa3e30e-kube-api-access-dxkzq\") pod \"node-resolver-rz46d\" (UID: \"d69db7df-9f0a-4339-99b0-1d7dffa3e30e\") " pod="openshift-dns/node-resolver-rz46d" Jul 08 19:42:00.912428 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912429 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-etc-kubernetes\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.912803 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912477 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-etc-modprobe-d\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.912803 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912504 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r7cv\" (UniqueName: \"kubernetes.io/projected/394cd92a-488b-4386-9c3c-5e607c321956-kube-api-access-5r7cv\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.912803 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912553 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-host-var-lib-cni-multus\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.912803 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912606 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-host-var-lib-kubelet\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.912803 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912616 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-etc-kubernetes\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.912803 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912639 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-multus-conf-dir\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.912803 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912665 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d69db7df-9f0a-4339-99b0-1d7dffa3e30e-hosts-file\") pod \"node-resolver-rz46d\" (UID: \"d69db7df-9f0a-4339-99b0-1d7dffa3e30e\") " pod="openshift-dns/node-resolver-rz46d" Jul 08 19:42:00.912803 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912687 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-etc-modprobe-d\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.912803 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912716 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-etc-systemd\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.912803 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912736 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-host-var-lib-kubelet\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.912803 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912761 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-etc-sysconfig\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.912803 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912776 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-multus-conf-dir\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.912803 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912785 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-lib-modules\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.912803 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912810 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/baeda2ff-0e38-4c4a-a057-dcd4e9020371-cni-binary-copy\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.913558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912825 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-etc-systemd\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.913558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912834 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-host\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.913558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912851 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-etc-sysconfig\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.913558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912860 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/394cd92a-488b-4386-9c3c-5e607c321956-etc-tuned\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.913558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912871 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d69db7df-9f0a-4339-99b0-1d7dffa3e30e-hosts-file\") pod \"node-resolver-rz46d\" (UID: \"d69db7df-9f0a-4339-99b0-1d7dffa3e30e\") " pod="openshift-dns/node-resolver-rz46d" Jul 08 19:42:00.913558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912884 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-hostroot\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.913558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912910 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs\") pod \"network-metrics-daemon-s7hbd\" (UID: \"3b5311a0-d5fa-4994-9071-d861d2a53b09\") " pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:00.913558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912940 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-host\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.913558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912953 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-etc-sysctl-conf\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.913558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912978 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-hostroot\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.913558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913026 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-var-lib-kubelet\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.913558 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:00.913082 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:42:00.913558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.912978 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-var-lib-kubelet\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.913558 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:00.913181 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs podName:3b5311a0-d5fa-4994-9071-d861d2a53b09 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:01.413159792 +0000 UTC m=+3.106469279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs") pod "network-metrics-daemon-s7hbd" (UID: "3b5311a0-d5fa-4994-9071-d861d2a53b09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:42:00.913558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913205 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-os-release\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.913558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913232 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-host-run-k8s-cni-cncf-io\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.913558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913269 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-sys\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.913558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913291 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-multus-cni-dir\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.914517 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913318 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-multus-socket-dir-parent\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.914517 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913344 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-host-var-lib-cni-bin\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.914517 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913342 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-lib-modules\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.914517 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913345 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-etc-sysctl-conf\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.914517 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913368 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-host-run-multus-certs\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.914517 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913416 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-host-run-k8s-cni-cncf-io\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.914517 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913436 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-multus-cni-dir\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.914517 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913480 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/baeda2ff-0e38-4c4a-a057-dcd4e9020371-cni-binary-copy\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.914517 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913491 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-multus-socket-dir-parent\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.914517 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913509 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-sys\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.914517 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913545 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwlcg\" (UniqueName: \"kubernetes.io/projected/baeda2ff-0e38-4c4a-a057-dcd4e9020371-kube-api-access-pwlcg\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.914517 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913565 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-host-run-multus-certs\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.914517 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913580 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/394cd92a-488b-4386-9c3c-5e607c321956-tmp\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.914517 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913607 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-system-cni-dir\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.914517 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913626 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d69db7df-9f0a-4339-99b0-1d7dffa3e30e-tmp-dir\") pod \"node-resolver-rz46d\" (UID: \"d69db7df-9f0a-4339-99b0-1d7dffa3e30e\") " pod="openshift-dns/node-resolver-rz46d" Jul 08 19:42:00.914517 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913620 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-os-release\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.914517 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913645 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-etc-sysctl-d\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.914517 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913673 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-host-var-lib-cni-bin\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.915487 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913683 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-run\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.915487 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913701 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-cnibin\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.915487 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913717 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/baeda2ff-0e38-4c4a-a057-dcd4e9020371-multus-daemon-config\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.915487 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913720 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-system-cni-dir\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.915487 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913828 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-run\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.915487 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913949 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d69db7df-9f0a-4339-99b0-1d7dffa3e30e-tmp-dir\") pod \"node-resolver-rz46d\" (UID: \"d69db7df-9f0a-4339-99b0-1d7dffa3e30e\") " pod="openshift-dns/node-resolver-rz46d" Jul 08 19:42:00.915487 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.913959 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/394cd92a-488b-4386-9c3c-5e607c321956-etc-sysctl-d\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.915487 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.914016 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/baeda2ff-0e38-4c4a-a057-dcd4e9020371-cnibin\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.915487 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.914483 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/baeda2ff-0e38-4c4a-a057-dcd4e9020371-multus-daemon-config\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:00.916056 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.916030 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/394cd92a-488b-4386-9c3c-5e607c321956-etc-tuned\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.916581 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.916553 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/394cd92a-488b-4386-9c3c-5e607c321956-tmp\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.921539 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.921507 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxkzq\" (UniqueName: \"kubernetes.io/projected/d69db7df-9f0a-4339-99b0-1d7dffa3e30e-kube-api-access-dxkzq\") pod \"node-resolver-rz46d\" (UID: \"d69db7df-9f0a-4339-99b0-1d7dffa3e30e\") " pod="openshift-dns/node-resolver-rz46d" Jul 08 19:42:00.921836 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.921814 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfdsk\" (UniqueName: \"kubernetes.io/projected/3b5311a0-d5fa-4994-9071-d861d2a53b09-kube-api-access-jfdsk\") pod \"network-metrics-daemon-s7hbd\" (UID: \"3b5311a0-d5fa-4994-9071-d861d2a53b09\") " pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:00.922365 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.922346 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r7cv\" (UniqueName: \"kubernetes.io/projected/394cd92a-488b-4386-9c3c-5e607c321956-kube-api-access-5r7cv\") pod \"tuned-m7wzx\" (UID: \"394cd92a-488b-4386-9c3c-5e607c321956\") " pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:00.922628 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:00.922602 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwlcg\" (UniqueName: \"kubernetes.io/projected/baeda2ff-0e38-4c4a-a057-dcd4e9020371-kube-api-access-pwlcg\") pod \"multus-h77hk\" (UID: \"baeda2ff-0e38-4c4a-a057-dcd4e9020371\") " pod="openshift-multus/multus-h77hk" Jul 08 19:42:01.005704 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.005429 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-279bc" Jul 08 19:42:01.012262 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.012232 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-g5gnj" Jul 08 19:42:01.017499 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:42:01.017468 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33ffe4eb_0ee2_4e34_a03b_b1f06bb98602.slice/crio-676d4dc6203a47de3f894a32b4d03d908962f7003f887a1d29780957c91f9a8e WatchSource:0}: Error finding container 676d4dc6203a47de3f894a32b4d03d908962f7003f887a1d29780957c91f9a8e: Status 404 returned error can't find the container with id 676d4dc6203a47de3f894a32b4d03d908962f7003f887a1d29780957c91f9a8e Jul 08 19:42:01.022862 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.022834 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xhjd9" Jul 08 19:42:01.024578 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:42:01.024544 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd277414e_55a2_411f_852d_aae01918e1c4.slice/crio-133d3b9a317ebc6523ecbf1e8466892805d0f95790c6c32ca87b5e032bf3cee8 WatchSource:0}: Error finding container 133d3b9a317ebc6523ecbf1e8466892805d0f95790c6c32ca87b5e032bf3cee8: Status 404 returned error can't find the container with id 133d3b9a317ebc6523ecbf1e8466892805d0f95790c6c32ca87b5e032bf3cee8 Jul 08 19:42:01.029318 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.029297 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:01.033270 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:42:01.033240 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d7662e_e8aa_4e30_ad77_7956f718b804.slice/crio-fa0185b9eaa81b5e3d6e6950be30d9bb018f7ab35fa456b800aac99bb8cabbd4 WatchSource:0}: Error finding container fa0185b9eaa81b5e3d6e6950be30d9bb018f7ab35fa456b800aac99bb8cabbd4: Status 404 returned error can't find the container with id fa0185b9eaa81b5e3d6e6950be30d9bb018f7ab35fa456b800aac99bb8cabbd4 Jul 08 19:42:01.039546 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:42:01.039520 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45d42bef_ca8b_42c5_8b2a_90485f01ce14.slice/crio-e686e34d0e853ce73f5e8fc0b8e2fa631d12d87f7ccaae829414937016f1fbab WatchSource:0}: Error finding container e686e34d0e853ce73f5e8fc0b8e2fa631d12d87f7ccaae829414937016f1fbab: Status 404 returned error can't find the container with id e686e34d0e853ce73f5e8fc0b8e2fa631d12d87f7ccaae829414937016f1fbab Jul 08 19:42:01.043211 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.043191 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-62sgn" Jul 08 19:42:01.053683 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.053656 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rz46d" Jul 08 19:42:01.054445 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:42:01.054417 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5cf071e_9c63_46a7_8fef_f1f11d2a43cd.slice/crio-9c1642451b9259d9a90c8c62806dd5326f996635f1f44601bcb9c6ea344876ab WatchSource:0}: Error finding container 9c1642451b9259d9a90c8c62806dd5326f996635f1f44601bcb9c6ea344876ab: Status 404 returned error can't find the container with id 9c1642451b9259d9a90c8c62806dd5326f996635f1f44601bcb9c6ea344876ab Jul 08 19:42:01.062296 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:42:01.062254 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd69db7df_9f0a_4339_99b0_1d7dffa3e30e.slice/crio-a71a5471b6d378d39ee5aee8fb09e9d2bc49ab7a025c49a656a20d64efcfbee5 WatchSource:0}: Error finding container a71a5471b6d378d39ee5aee8fb09e9d2bc49ab7a025c49a656a20d64efcfbee5: Status 404 returned error can't find the container with id a71a5471b6d378d39ee5aee8fb09e9d2bc49ab7a025c49a656a20d64efcfbee5 Jul 08 19:42:01.063773 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.063558 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" Jul 08 19:42:01.071618 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.071593 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" Jul 08 19:42:01.074060 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:42:01.074023 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccc6206f_2d22_4d59_b298_4fa3d7aec4da.slice/crio-9b1a17357f805e14a7108283e7bc132a5e53abf8bdf2d61b7d482b6157bac894 WatchSource:0}: Error finding container 9b1a17357f805e14a7108283e7bc132a5e53abf8bdf2d61b7d482b6157bac894: Status 404 returned error can't find the container with id 9b1a17357f805e14a7108283e7bc132a5e53abf8bdf2d61b7d482b6157bac894 Jul 08 19:42:01.079770 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.079750 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h77hk" Jul 08 19:42:01.080759 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:42:01.080734 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod394cd92a_488b_4386_9c3c_5e607c321956.slice/crio-4f425e03e1f83a0b147c748dee2061793efa45e0fdae78c0804901fa72205720 WatchSource:0}: Error finding container 4f425e03e1f83a0b147c748dee2061793efa45e0fdae78c0804901fa72205720: Status 404 returned error can't find the container with id 4f425e03e1f83a0b147c748dee2061793efa45e0fdae78c0804901fa72205720 Jul 08 19:42:01.091432 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:42:01.091399 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaeda2ff_0e38_4c4a_a057_dcd4e9020371.slice/crio-1d9e731e53bc397628380e8a33610f3d5950f848c45ff6525baae83e661e90dc WatchSource:0}: Error finding container 1d9e731e53bc397628380e8a33610f3d5950f848c45ff6525baae83e661e90dc: Status 404 returned error can't find the container with id 1d9e731e53bc397628380e8a33610f3d5950f848c45ff6525baae83e661e90dc Jul 08 19:42:01.317332 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.317292 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret\") pod \"global-pull-secret-syncer-82lzg\" (UID: \"3184c318-9df8-4f30-92e9-5bf8786149e9\") " pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:01.317532 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:01.317491 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:42:01.317597 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:01.317581 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret podName:3184c318-9df8-4f30-92e9-5bf8786149e9 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:02.317556572 +0000 UTC m=+4.010866044 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret") pod "global-pull-secret-syncer-82lzg" (UID: "3184c318-9df8-4f30-92e9-5bf8786149e9") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:42:01.418732 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.418692 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs\") pod \"network-metrics-daemon-s7hbd\" (UID: \"3b5311a0-d5fa-4994-9071-d861d2a53b09\") " pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:01.418941 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.418750 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89clm\" (UniqueName: \"kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm\") pod \"network-check-target-q6qvd\" (UID: \"dc75384f-fbaa-4ddf-b258-02a4024686aa\") " pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:01.418941 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:01.418868 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:42:01.418941 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:01.418890 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:42:01.418941 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:01.418910 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:42:01.418941 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:01.418923 2568 projected.go:196] Error preparing data for projected volume kube-api-access-89clm for pod openshift-network-diagnostics/network-check-target-q6qvd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:42:01.419249 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:01.418951 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs podName:3b5311a0-d5fa-4994-9071-d861d2a53b09 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:02.418934408 +0000 UTC m=+4.112243877 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs") pod "network-metrics-daemon-s7hbd" (UID: "3b5311a0-d5fa-4994-9071-d861d2a53b09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:42:01.419249 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:01.418972 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm podName:dc75384f-fbaa-4ddf-b258-02a4024686aa nodeName:}" failed. No retries permitted until 2026-07-08 19:42:02.41895982 +0000 UTC m=+4.112269286 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-89clm" (UniqueName: "kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm") pod "network-check-target-q6qvd" (UID: "dc75384f-fbaa-4ddf-b258-02a4024686aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:42:01.758057 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.757930 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 19:37:01 +0000 UTC" deadline="2028-02-21 11:06:42.815197598 +0000 UTC" Jul 08 19:42:01.758057 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.757982 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14223h24m41.057218027s" Jul 08 19:42:01.834310 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.834271 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" event={"ID":"394cd92a-488b-4386-9c3c-5e607c321956","Type":"ContainerStarted","Data":"4f425e03e1f83a0b147c748dee2061793efa45e0fdae78c0804901fa72205720"} Jul 08 19:42:01.835577 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.835544 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" event={"ID":"ccc6206f-2d22-4d59-b298-4fa3d7aec4da","Type":"ContainerStarted","Data":"9b1a17357f805e14a7108283e7bc132a5e53abf8bdf2d61b7d482b6157bac894"} Jul 08 19:42:01.836690 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.836666 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g5gnj" event={"ID":"d277414e-55a2-411f-852d-aae01918e1c4","Type":"ContainerStarted","Data":"133d3b9a317ebc6523ecbf1e8466892805d0f95790c6c32ca87b5e032bf3cee8"} Jul 08 19:42:01.837854 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.837825 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h77hk" event={"ID":"baeda2ff-0e38-4c4a-a057-dcd4e9020371","Type":"ContainerStarted","Data":"1d9e731e53bc397628380e8a33610f3d5950f848c45ff6525baae83e661e90dc"} Jul 08 19:42:01.838998 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.838969 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rz46d" event={"ID":"d69db7df-9f0a-4339-99b0-1d7dffa3e30e","Type":"ContainerStarted","Data":"a71a5471b6d378d39ee5aee8fb09e9d2bc49ab7a025c49a656a20d64efcfbee5"} Jul 08 19:42:01.840115 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.840090 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-62sgn" event={"ID":"a5cf071e-9c63-46a7-8fef-f1f11d2a43cd","Type":"ContainerStarted","Data":"9c1642451b9259d9a90c8c62806dd5326f996635f1f44601bcb9c6ea344876ab"} Jul 08 19:42:01.841392 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.841370 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" event={"ID":"45d42bef-ca8b-42c5-8b2a-90485f01ce14","Type":"ContainerStarted","Data":"e686e34d0e853ce73f5e8fc0b8e2fa631d12d87f7ccaae829414937016f1fbab"} Jul 08 19:42:01.843326 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.843293 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xhjd9" event={"ID":"f6d7662e-e8aa-4e30-ad77-7956f718b804","Type":"ContainerStarted","Data":"fa0185b9eaa81b5e3d6e6950be30d9bb018f7ab35fa456b800aac99bb8cabbd4"} Jul 08 19:42:01.844498 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:01.844477 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-279bc" event={"ID":"33ffe4eb-0ee2-4e34-a03b-b1f06bb98602","Type":"ContainerStarted","Data":"676d4dc6203a47de3f894a32b4d03d908962f7003f887a1d29780957c91f9a8e"} Jul 08 19:42:02.327478 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:02.327432 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret\") pod \"global-pull-secret-syncer-82lzg\" (UID: \"3184c318-9df8-4f30-92e9-5bf8786149e9\") " pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:02.327673 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:02.327648 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:42:02.327743 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:02.327715 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret podName:3184c318-9df8-4f30-92e9-5bf8786149e9 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:04.32769633 +0000 UTC m=+6.021005803 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret") pod "global-pull-secret-syncer-82lzg" (UID: "3184c318-9df8-4f30-92e9-5bf8786149e9") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:42:02.428514 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:02.428472 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs\") pod \"network-metrics-daemon-s7hbd\" (UID: \"3b5311a0-d5fa-4994-9071-d861d2a53b09\") " pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:02.428718 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:02.428559 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89clm\" (UniqueName: \"kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm\") pod \"network-check-target-q6qvd\" (UID: \"dc75384f-fbaa-4ddf-b258-02a4024686aa\") " pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:02.428718 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:02.428657 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:42:02.428841 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:02.428731 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:42:02.428841 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:02.428740 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs podName:3b5311a0-d5fa-4994-9071-d861d2a53b09 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:04.428717853 +0000 UTC m=+6.122027335 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs") pod "network-metrics-daemon-s7hbd" (UID: "3b5311a0-d5fa-4994-9071-d861d2a53b09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:42:02.428841 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:02.428752 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:42:02.428841 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:02.428768 2568 projected.go:196] Error preparing data for projected volume kube-api-access-89clm for pod openshift-network-diagnostics/network-check-target-q6qvd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:42:02.429052 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:02.428907 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm podName:dc75384f-fbaa-4ddf-b258-02a4024686aa nodeName:}" failed. No retries permitted until 2026-07-08 19:42:04.428889759 +0000 UTC m=+6.122199250 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-89clm" (UniqueName: "kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm") pod "network-check-target-q6qvd" (UID: "dc75384f-fbaa-4ddf-b258-02a4024686aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:42:02.758949 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:02.758801 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 19:37:01 +0000 UTC" deadline="2028-01-17 00:36:28.580460407 +0000 UTC" Jul 08 19:42:02.758949 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:02.758839 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13372h54m25.821626469s" Jul 08 19:42:02.830078 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:02.829229 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:02.830078 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:02.829365 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82lzg" podUID="3184c318-9df8-4f30-92e9-5bf8786149e9" Jul 08 19:42:02.830078 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:02.829777 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:02.830078 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:02.829868 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6qvd" podUID="dc75384f-fbaa-4ddf-b258-02a4024686aa" Jul 08 19:42:02.830078 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:02.829932 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:02.830078 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:02.830019 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:42:04.347765 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:04.347727 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret\") pod \"global-pull-secret-syncer-82lzg\" (UID: \"3184c318-9df8-4f30-92e9-5bf8786149e9\") " pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:04.348313 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:04.347927 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:42:04.348313 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:04.347995 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret podName:3184c318-9df8-4f30-92e9-5bf8786149e9 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:08.347976288 +0000 UTC m=+10.041285770 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret") pod "global-pull-secret-syncer-82lzg" (UID: "3184c318-9df8-4f30-92e9-5bf8786149e9") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:42:04.449570 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:04.448687 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs\") pod \"network-metrics-daemon-s7hbd\" (UID: \"3b5311a0-d5fa-4994-9071-d861d2a53b09\") " pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:04.449570 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:04.448757 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89clm\" (UniqueName: \"kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm\") pod \"network-check-target-q6qvd\" (UID: \"dc75384f-fbaa-4ddf-b258-02a4024686aa\") " pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:04.449570 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:04.448924 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:42:04.449570 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:04.448943 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:42:04.449570 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:04.448958 2568 projected.go:196] Error preparing data for projected volume kube-api-access-89clm for pod openshift-network-diagnostics/network-check-target-q6qvd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:42:04.449570 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:04.449019 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm podName:dc75384f-fbaa-4ddf-b258-02a4024686aa nodeName:}" failed. No retries permitted until 2026-07-08 19:42:08.448999344 +0000 UTC m=+10.142308813 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-89clm" (UniqueName: "kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm") pod "network-check-target-q6qvd" (UID: "dc75384f-fbaa-4ddf-b258-02a4024686aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:42:04.449570 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:04.449459 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:42:04.449570 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:04.449512 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs podName:3b5311a0-d5fa-4994-9071-d861d2a53b09 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:08.449495083 +0000 UTC m=+10.142804568 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs") pod "network-metrics-daemon-s7hbd" (UID: "3b5311a0-d5fa-4994-9071-d861d2a53b09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:42:04.826496 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:04.826460 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:04.826693 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:04.826598 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82lzg" podUID="3184c318-9df8-4f30-92e9-5bf8786149e9" Jul 08 19:42:04.827055 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:04.827028 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:04.827178 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:04.827156 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6qvd" podUID="dc75384f-fbaa-4ddf-b258-02a4024686aa" Jul 08 19:42:04.827340 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:04.827323 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:04.827447 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:04.827427 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:42:06.826085 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:06.826051 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:06.826575 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:06.826218 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82lzg" podUID="3184c318-9df8-4f30-92e9-5bf8786149e9" Jul 08 19:42:06.826575 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:06.826059 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:06.826575 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:06.826348 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:42:06.826575 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:06.826052 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:06.826575 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:06.826434 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6qvd" podUID="dc75384f-fbaa-4ddf-b258-02a4024686aa" Jul 08 19:42:08.384300 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:08.384255 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret\") pod \"global-pull-secret-syncer-82lzg\" (UID: \"3184c318-9df8-4f30-92e9-5bf8786149e9\") " pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:08.384780 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:08.384479 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:42:08.384780 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:08.384571 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret podName:3184c318-9df8-4f30-92e9-5bf8786149e9 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:16.384547464 +0000 UTC m=+18.077856951 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret") pod "global-pull-secret-syncer-82lzg" (UID: "3184c318-9df8-4f30-92e9-5bf8786149e9") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:42:08.484843 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:08.484795 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs\") pod \"network-metrics-daemon-s7hbd\" (UID: \"3b5311a0-d5fa-4994-9071-d861d2a53b09\") " pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:08.485045 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:08.484857 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89clm\" (UniqueName: \"kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm\") pod \"network-check-target-q6qvd\" (UID: \"dc75384f-fbaa-4ddf-b258-02a4024686aa\") " pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:08.485045 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:08.485000 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:42:08.485045 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:08.485009 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:42:08.485045 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:08.485100 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs podName:3b5311a0-d5fa-4994-9071-d861d2a53b09 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:16.485077013 +0000 UTC m=+18.178386485 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs") pod "network-metrics-daemon-s7hbd" (UID: "3b5311a0-d5fa-4994-9071-d861d2a53b09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:42:08.485045 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:08.485131 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:42:08.485045 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:08.485151 2568 projected.go:196] Error preparing data for projected volume kube-api-access-89clm for pod openshift-network-diagnostics/network-check-target-q6qvd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:42:08.485045 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:08.485208 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm podName:dc75384f-fbaa-4ddf-b258-02a4024686aa nodeName:}" failed. No retries permitted until 2026-07-08 19:42:16.485192031 +0000 UTC m=+18.178501516 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-89clm" (UniqueName: "kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm") pod "network-check-target-q6qvd" (UID: "dc75384f-fbaa-4ddf-b258-02a4024686aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:42:08.826909 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:08.826868 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:08.827130 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:08.826982 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:08.827130 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:08.826988 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82lzg" podUID="3184c318-9df8-4f30-92e9-5bf8786149e9" Jul 08 19:42:08.827130 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:08.827088 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6qvd" podUID="dc75384f-fbaa-4ddf-b258-02a4024686aa" Jul 08 19:42:08.827130 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:08.827101 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:08.827342 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:08.827240 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:42:10.826617 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:10.826586 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:10.827067 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:10.826595 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:10.827067 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:10.826713 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:42:10.827067 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:10.826769 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:10.827067 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:10.826871 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82lzg" podUID="3184c318-9df8-4f30-92e9-5bf8786149e9" Jul 08 19:42:10.827067 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:10.826934 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6qvd" podUID="dc75384f-fbaa-4ddf-b258-02a4024686aa" Jul 08 19:42:12.826562 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:12.826088 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:12.826562 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:12.826107 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:12.826562 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:12.826093 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:12.826562 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:12.826233 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:42:12.826562 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:12.826335 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82lzg" podUID="3184c318-9df8-4f30-92e9-5bf8786149e9" Jul 08 19:42:12.826562 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:12.826415 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6qvd" podUID="dc75384f-fbaa-4ddf-b258-02a4024686aa" Jul 08 19:42:14.825975 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:14.825940 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:14.826421 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:14.825944 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:14.826421 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:14.826089 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6qvd" podUID="dc75384f-fbaa-4ddf-b258-02a4024686aa" Jul 08 19:42:14.826421 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:14.825944 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:14.826421 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:14.826161 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82lzg" podUID="3184c318-9df8-4f30-92e9-5bf8786149e9" Jul 08 19:42:14.826421 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:14.826262 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:42:16.444590 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:16.444553 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret\") pod \"global-pull-secret-syncer-82lzg\" (UID: \"3184c318-9df8-4f30-92e9-5bf8786149e9\") " pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:16.444996 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:16.444673 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:42:16.444996 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:16.444732 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret podName:3184c318-9df8-4f30-92e9-5bf8786149e9 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:32.444717127 +0000 UTC m=+34.138026591 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret") pod "global-pull-secret-syncer-82lzg" (UID: "3184c318-9df8-4f30-92e9-5bf8786149e9") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:42:16.544894 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:16.544858 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs\") pod \"network-metrics-daemon-s7hbd\" (UID: \"3b5311a0-d5fa-4994-9071-d861d2a53b09\") " pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:16.544894 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:16.544899 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89clm\" (UniqueName: \"kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm\") pod \"network-check-target-q6qvd\" (UID: \"dc75384f-fbaa-4ddf-b258-02a4024686aa\") " pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:16.545130 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:16.545011 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:42:16.545130 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:16.545091 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:42:16.545130 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:16.545103 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs podName:3b5311a0-d5fa-4994-9071-d861d2a53b09 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:32.545083058 +0000 UTC m=+34.238392537 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs") pod "network-metrics-daemon-s7hbd" (UID: "3b5311a0-d5fa-4994-9071-d861d2a53b09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:42:16.545130 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:16.545109 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:42:16.545130 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:16.545122 2568 projected.go:196] Error preparing data for projected volume kube-api-access-89clm for pod openshift-network-diagnostics/network-check-target-q6qvd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:42:16.545346 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:16.545156 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm podName:dc75384f-fbaa-4ddf-b258-02a4024686aa nodeName:}" failed. No retries permitted until 2026-07-08 19:42:32.545145309 +0000 UTC m=+34.238454787 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-89clm" (UniqueName: "kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm") pod "network-check-target-q6qvd" (UID: "dc75384f-fbaa-4ddf-b258-02a4024686aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:42:16.826667 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:16.826632 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:16.826848 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:16.826637 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:16.826848 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:16.826776 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:42:16.826848 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:16.826638 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:16.826848 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:16.826811 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6qvd" podUID="dc75384f-fbaa-4ddf-b258-02a4024686aa" Jul 08 19:42:16.827068 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:16.826915 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82lzg" podUID="3184c318-9df8-4f30-92e9-5bf8786149e9" Jul 08 19:42:18.827081 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:18.827025 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:18.827531 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:18.827092 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:18.827531 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:18.827160 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:42:18.827531 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:18.827224 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:18.827531 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:18.827300 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82lzg" podUID="3184c318-9df8-4f30-92e9-5bf8786149e9" Jul 08 19:42:18.827531 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:18.827394 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6qvd" podUID="dc75384f-fbaa-4ddf-b258-02a4024686aa" Jul 08 19:42:20.827283 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:20.826892 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:20.827818 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:20.827386 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:42:20.827818 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:20.827233 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:20.827818 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:20.827084 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:20.827818 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:20.827485 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6qvd" podUID="dc75384f-fbaa-4ddf-b258-02a4024686aa" Jul 08 19:42:20.827818 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:20.827552 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82lzg" podUID="3184c318-9df8-4f30-92e9-5bf8786149e9" Jul 08 19:42:20.895723 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:20.895519 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-198.ec2.internal" event={"ID":"4518754363eeef80e023637afe5321de","Type":"ContainerStarted","Data":"5ded6cc888ec6df219ffbce18cddd39e316a4078d724e69211a9b5e78df42e57"} Jul 08 19:42:20.900139 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:20.899854 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h77hk" event={"ID":"baeda2ff-0e38-4c4a-a057-dcd4e9020371","Type":"ContainerStarted","Data":"cf4ab7d7df1680ec5097fe064e9033e713d0b7de47488326bfc7af2b9b5b96f9"} Jul 08 19:42:20.909807 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:20.909695 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-198.ec2.internal" podStartSLOduration=20.90967952 podStartE2EDuration="20.90967952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:42:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:20.908720284 +0000 UTC m=+22.602029767" watchObservedRunningTime="2026-07-08 19:42:20.90967952 +0000 UTC m=+22.602989007" Jul 08 19:42:20.912289 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:20.911860 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" event={"ID":"45d42bef-ca8b-42c5-8b2a-90485f01ce14","Type":"ContainerStarted","Data":"efef37a1a25966497bffcfa1413637efb35cf0dc0c6eb632302eab6d1629d9b0"} Jul 08 19:42:20.912289 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:20.911955 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" event={"ID":"45d42bef-ca8b-42c5-8b2a-90485f01ce14","Type":"ContainerStarted","Data":"57989b62563840734fd21b20c51e58d3877266183f292ec6407096aea1830f27"} Jul 08 19:42:20.916564 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:20.916004 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" event={"ID":"394cd92a-488b-4386-9c3c-5e607c321956","Type":"ContainerStarted","Data":"c4249e06afbd6de73378950051cc9119964af9974c3d9299fb27c30465730e22"} Jul 08 19:42:20.927260 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:20.926969 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-h77hk" podStartSLOduration=2.379150954 podStartE2EDuration="21.926948115s" totalImagesPullingTime="19.547797161s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:20.926250678 +0000 UTC m=+22.619560165" watchObservedRunningTime="2026-07-08 19:42:20.926948115 +0000 UTC m=+22.620257603" Jul 08 19:42:20.945924 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:20.945870 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-m7wzx" podStartSLOduration=2.435468816 podStartE2EDuration="21.945848958s" totalImagesPullingTime="19.510380142s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:20.94532024 +0000 UTC m=+22.638629725" watchObservedRunningTime="2026-07-08 19:42:20.945848958 +0000 UTC m=+22.639158445" Jul 08 19:42:21.920075 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:21.919976 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" event={"ID":"ccc6206f-2d22-4d59-b298-4fa3d7aec4da","Type":"ContainerStarted","Data":"52cb58ae8314656a9e04a8cec68724c9e686a9d7d2b67c5290ace442abfe8f87"} Jul 08 19:42:21.921700 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:21.921676 2568 generic.go:356] "Generic (PLEG): container finished" podID="d277414e-55a2-411f-852d-aae01918e1c4" containerID="7a9d9efabecd32e099ae1e7164a52f3b74722623379d158ffa3bc2c24a944be1" exitCode=0 Jul 08 19:42:21.921842 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:21.921765 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g5gnj" event={"ID":"d277414e-55a2-411f-852d-aae01918e1c4","Type":"ContainerDied","Data":"7a9d9efabecd32e099ae1e7164a52f3b74722623379d158ffa3bc2c24a944be1"} Jul 08 19:42:21.923378 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:21.923287 2568 generic.go:356] "Generic (PLEG): container finished" podID="3ae59f7384267d6a13c3e95812346171" containerID="a445ef060b59017dc640d20f071537bd97cafe67d8601d6398c0b4f16882072d" exitCode=0 Jul 08 19:42:21.923378 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:21.923315 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal" event={"ID":"3ae59f7384267d6a13c3e95812346171","Type":"ContainerDied","Data":"a445ef060b59017dc640d20f071537bd97cafe67d8601d6398c0b4f16882072d"} Jul 08 19:42:21.924905 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:21.924867 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rz46d" event={"ID":"d69db7df-9f0a-4339-99b0-1d7dffa3e30e","Type":"ContainerStarted","Data":"2bacb086308dba8236579d1f29551ee7359d59f1fbe375de50ace28a2e8b56b4"} Jul 08 19:42:21.926448 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:21.926424 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-62sgn" event={"ID":"a5cf071e-9c63-46a7-8fef-f1f11d2a43cd","Type":"ContainerStarted","Data":"127d413cd11231b4760dc7f137a807bb784ddc4524b671a476276df37210e72b"} Jul 08 19:42:21.931760 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:21.931734 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" event={"ID":"45d42bef-ca8b-42c5-8b2a-90485f01ce14","Type":"ContainerStarted","Data":"0228835da88e0f5bde3060f582193666409e9231c10ac287fea5c1b86ef671b3"} Jul 08 19:42:21.931861 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:21.931817 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" event={"ID":"45d42bef-ca8b-42c5-8b2a-90485f01ce14","Type":"ContainerStarted","Data":"e581b8b8bbefdaf3c08f5d8b34e66258a4c1dcab7936b194a0e9a6871e5436fb"} Jul 08 19:42:21.931922 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:21.931866 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" event={"ID":"45d42bef-ca8b-42c5-8b2a-90485f01ce14","Type":"ContainerStarted","Data":"b7c3d554a8ece6c2e9d2ebdc0de191c110eeb173c1657d8ec299043533f5929d"} Jul 08 19:42:21.931922 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:21.931911 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" event={"ID":"45d42bef-ca8b-42c5-8b2a-90485f01ce14","Type":"ContainerStarted","Data":"f80606e170545f2c64cdcda484548ea1a75f0b119ea75b22814fa5d36bd1570e"} Jul 08 19:42:21.934550 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:21.934513 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xhjd9" event={"ID":"f6d7662e-e8aa-4e30-ad77-7956f718b804","Type":"ContainerStarted","Data":"8fcb2e4d3236a70861b9335da718f8b00bf844dae44e5adf57a904e2dd3b62aa"} Jul 08 19:42:21.936418 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:21.936393 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-279bc" event={"ID":"33ffe4eb-0ee2-4e34-a03b-b1f06bb98602","Type":"ContainerStarted","Data":"02b6d0a8d4475cca4b38c92b73f337b8add135b97094508ee1f87e3cbf6fcc14"} Jul 08 19:42:21.976471 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:21.976413 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xhjd9" podStartSLOduration=3.475058965 podStartE2EDuration="22.976394195s" totalImagesPullingTime="19.50133523s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:21.975880788 +0000 UTC m=+23.669190275" watchObservedRunningTime="2026-07-08 19:42:21.976394195 +0000 UTC m=+23.669703683" Jul 08 19:42:21.990289 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:21.990239 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-62sgn" podStartSLOduration=3.453815708 podStartE2EDuration="22.990224355s" totalImagesPullingTime="19.536408647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:21.989884234 +0000 UTC m=+23.683193722" watchObservedRunningTime="2026-07-08 19:42:21.990224355 +0000 UTC m=+23.683533841" Jul 08 19:42:22.004337 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:22.004295 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rz46d" podStartSLOduration=12.980633358 podStartE2EDuration="23.004273414s" totalImagesPullingTime="10.023640056s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:22.004125309 +0000 UTC m=+23.697434796" watchObservedRunningTime="2026-07-08 19:42:22.004273414 +0000 UTC m=+23.697582901" Jul 08 19:42:22.019431 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:22.019379 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-279bc" podStartSLOduration=12.948811544 podStartE2EDuration="23.019361464s" totalImagesPullingTime="10.07054992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:22.018976641 +0000 UTC m=+23.712286129" watchObservedRunningTime="2026-07-08 19:42:22.019361464 +0000 UTC m=+23.712670990" Jul 08 19:42:22.090521 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:22.090487 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-279bc" Jul 08 19:42:22.091195 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:22.091177 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-279bc" Jul 08 19:42:22.431860 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:22.431672 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 19:42:22.768347 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:22.768178 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T19:42:22.431822161Z","UUID":"d321a494-1273-4833-83b7-e7d1610460a7","Handler":null,"Name":"","Endpoint":""} Jul 08 19:42:22.769874 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:22.769840 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 19:42:22.769993 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:22.769891 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 19:42:22.825966 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:22.825937 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:22.826164 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:22.825937 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:22.826164 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:22.826082 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:22.826164 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:22.826086 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82lzg" podUID="3184c318-9df8-4f30-92e9-5bf8786149e9" Jul 08 19:42:22.826340 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:22.826190 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:42:22.826340 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:22.826273 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6qvd" podUID="dc75384f-fbaa-4ddf-b258-02a4024686aa" Jul 08 19:42:22.941252 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:22.941220 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" event={"ID":"ccc6206f-2d22-4d59-b298-4fa3d7aec4da","Type":"ContainerStarted","Data":"0e2037e9269ac6885efc3209f4b41010edd5ae1b09b88b52d48de040948e6df5"} Jul 08 19:42:22.943338 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:22.943306 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal" event={"ID":"3ae59f7384267d6a13c3e95812346171","Type":"ContainerStarted","Data":"b179f4805a89d93366f00f06e2c3e62c7ea71d3aee12621c8c3431c494932c13"} Jul 08 19:42:22.943927 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:22.943545 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-279bc" Jul 08 19:42:22.944444 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:22.944425 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-279bc" Jul 08 19:42:22.959917 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:22.959865 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-198.ec2.internal" podStartSLOduration=22.959846519 podStartE2EDuration="22.959846519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:42:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:22.959482995 +0000 UTC m=+24.652792482" watchObservedRunningTime="2026-07-08 19:42:22.959846519 +0000 UTC m=+24.653156007" Jul 08 19:42:23.948865 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:23.948782 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" event={"ID":"45d42bef-ca8b-42c5-8b2a-90485f01ce14","Type":"ContainerStarted","Data":"3a6966f24386242e3701b8bd6823dea3cef68085e725960edb11b175f148f6c3"} Jul 08 19:42:23.950657 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:23.950628 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" event={"ID":"ccc6206f-2d22-4d59-b298-4fa3d7aec4da","Type":"ContainerStarted","Data":"e8b02f0e45a8ea69daf08608bda03fc904f7ee76e6a911fccf5c206af772205b"} Jul 08 19:42:23.969337 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:23.969296 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phpxv" podStartSLOduration=3.007946416 podStartE2EDuration="24.969281195s" totalImagesPullingTime="21.961334779s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:23.967942361 +0000 UTC m=+25.661251848" watchObservedRunningTime="2026-07-08 19:42:23.969281195 +0000 UTC m=+25.662590681" Jul 08 19:42:24.186191 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:24.186145 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:24.275005 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:24.274925 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:24.826112 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:24.826082 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:24.826294 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:24.826080 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:24.826294 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:24.826201 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:42:24.826294 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:24.826091 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:24.826294 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:24.826283 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6qvd" podUID="dc75384f-fbaa-4ddf-b258-02a4024686aa" Jul 08 19:42:24.826475 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:24.826349 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82lzg" podUID="3184c318-9df8-4f30-92e9-5bf8786149e9" Jul 08 19:42:25.958545 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:25.958349 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" event={"ID":"45d42bef-ca8b-42c5-8b2a-90485f01ce14","Type":"ContainerStarted","Data":"bf8b4e75151619000f196e2a435140fdaf46725549be02062aead213e863cb4c"} Jul 08 19:42:25.958996 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:25.958609 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:25.958996 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:25.958651 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:25.958996 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:25.958694 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:25.974681 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:25.974650 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:25.974908 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:25.974891 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:25.986075 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:25.986016 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" podStartSLOduration=7.366672199 podStartE2EDuration="26.986004447s" totalImagesPullingTime="19.619332248s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:25.985603403 +0000 UTC m=+27.678912926" watchObservedRunningTime="2026-07-08 19:42:25.986004447 +0000 UTC m=+27.679313958" Jul 08 19:42:26.829473 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:26.828819 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:26.829473 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:26.828948 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82lzg" podUID="3184c318-9df8-4f30-92e9-5bf8786149e9" Jul 08 19:42:26.829473 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:26.829320 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:26.829473 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:26.829405 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6qvd" podUID="dc75384f-fbaa-4ddf-b258-02a4024686aa" Jul 08 19:42:26.829473 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:26.829449 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:26.829831 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:26.829524 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:42:27.498664 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:27.498621 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-82lzg"] Jul 08 19:42:27.499435 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:27.498788 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:27.499435 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:27.498874 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82lzg" podUID="3184c318-9df8-4f30-92e9-5bf8786149e9" Jul 08 19:42:27.502674 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:27.502641 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s7hbd"] Jul 08 19:42:27.502796 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:27.502785 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:27.502926 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:27.502900 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:42:27.503386 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:27.503363 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q6qvd"] Jul 08 19:42:27.503479 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:27.503467 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:27.503577 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:27.503555 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6qvd" podUID="dc75384f-fbaa-4ddf-b258-02a4024686aa" Jul 08 19:42:28.828447 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:28.828422 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:28.828815 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:28.828508 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6qvd" podUID="dc75384f-fbaa-4ddf-b258-02a4024686aa" Jul 08 19:42:29.826147 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:29.826121 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:29.826383 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:29.826229 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:42:29.826383 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:29.826280 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:29.826383 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:29.826377 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82lzg" podUID="3184c318-9df8-4f30-92e9-5bf8786149e9" Jul 08 19:42:29.968573 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:29.968485 2568 generic.go:356] "Generic (PLEG): container finished" podID="d277414e-55a2-411f-852d-aae01918e1c4" containerID="47abc493c6fc8b7f90058faa362aea462b56e5d4ce79878a1e11acfd9f834617" exitCode=0 Jul 08 19:42:29.968573 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:29.968551 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g5gnj" event={"ID":"d277414e-55a2-411f-852d-aae01918e1c4","Type":"ContainerDied","Data":"47abc493c6fc8b7f90058faa362aea462b56e5d4ce79878a1e11acfd9f834617"} Jul 08 19:42:30.826587 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:30.826555 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:30.826753 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:30.826685 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6qvd" podUID="dc75384f-fbaa-4ddf-b258-02a4024686aa" Jul 08 19:42:31.026352 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:31.026318 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:31.120944 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:31.120860 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:31.826284 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:31.826252 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:31.826422 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:31.826295 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:31.826422 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:31.826364 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:42:31.826483 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:31.826466 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82lzg" podUID="3184c318-9df8-4f30-92e9-5bf8786149e9" Jul 08 19:42:31.974764 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:31.974735 2568 generic.go:356] "Generic (PLEG): container finished" podID="d277414e-55a2-411f-852d-aae01918e1c4" containerID="11e6289b023e234fba1d0f99d04b57a2bf993b47b6e9c1236c4d5a8462b5531a" exitCode=0 Jul 08 19:42:31.974932 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:31.974772 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g5gnj" event={"ID":"d277414e-55a2-411f-852d-aae01918e1c4","Type":"ContainerDied","Data":"11e6289b023e234fba1d0f99d04b57a2bf993b47b6e9c1236c4d5a8462b5531a"} Jul 08 19:42:32.135913 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:32.135807 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:32.235686 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:32.235653 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:32.336722 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:32.336686 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:32.457005 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:32.456914 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret\") pod \"global-pull-secret-syncer-82lzg\" (UID: \"3184c318-9df8-4f30-92e9-5bf8786149e9\") " pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:32.457190 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:32.457165 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:42:32.457259 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:32.457246 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret podName:3184c318-9df8-4f30-92e9-5bf8786149e9 nodeName:}" failed. No retries permitted until 2026-07-08 19:43:04.457225111 +0000 UTC m=+66.150534579 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret") pod "global-pull-secret-syncer-82lzg" (UID: "3184c318-9df8-4f30-92e9-5bf8786149e9") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:42:32.557838 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:32.557807 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs\") pod \"network-metrics-daemon-s7hbd\" (UID: \"3b5311a0-d5fa-4994-9071-d861d2a53b09\") " pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:32.557838 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:32.557843 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89clm\" (UniqueName: \"kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm\") pod \"network-check-target-q6qvd\" (UID: \"dc75384f-fbaa-4ddf-b258-02a4024686aa\") " pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:32.558075 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:32.557946 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:42:32.558075 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:32.557959 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:42:32.558075 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:32.557969 2568 projected.go:196] Error preparing data for projected volume kube-api-access-89clm for pod openshift-network-diagnostics/network-check-target-q6qvd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:42:32.558075 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:32.557974 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:42:32.558075 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:32.558016 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm podName:dc75384f-fbaa-4ddf-b258-02a4024686aa nodeName:}" failed. No retries permitted until 2026-07-08 19:43:04.558002786 +0000 UTC m=+66.251312250 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-89clm" (UniqueName: "kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm") pod "network-check-target-q6qvd" (UID: "dc75384f-fbaa-4ddf-b258-02a4024686aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:42:32.558075 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:32.558060 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs podName:3b5311a0-d5fa-4994-9071-d861d2a53b09 nodeName:}" failed. No retries permitted until 2026-07-08 19:43:04.558023451 +0000 UTC m=+66.251332931 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs") pod "network-metrics-daemon-s7hbd" (UID: "3b5311a0-d5fa-4994-9071-d861d2a53b09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:42:32.826388 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:32.826363 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:32.826560 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:32.826490 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6qvd" podUID="dc75384f-fbaa-4ddf-b258-02a4024686aa" Jul 08 19:42:33.103760 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.103728 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:33.170379 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.170332 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:33.226082 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.226030 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:33.630933 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.630908 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-198.ec2.internal" event="NodeReady" Jul 08 19:42:33.631199 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.631020 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 19:42:33.674000 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.673969 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vppxs"] Jul 08 19:42:33.693437 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.693410 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bvr92"] Jul 08 19:42:33.693565 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.693537 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vppxs" Jul 08 19:42:33.696326 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.696305 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 19:42:33.696463 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.696307 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4qkct\"" Jul 08 19:42:33.696463 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.696455 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 19:42:33.696583 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.696523 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 19:42:33.717511 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.717487 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vppxs"] Jul 08 19:42:33.717511 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.717519 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bvr92"] Jul 08 19:42:33.717668 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.717598 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bvr92" Jul 08 19:42:33.720363 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.720345 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w5s5j\"" Jul 08 19:42:33.720473 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.720347 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 19:42:33.720564 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.720553 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 19:42:33.767577 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.767545 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8h7r\" (UniqueName: \"kubernetes.io/projected/faf3ad2e-6304-49c5-be6e-560a500240f8-kube-api-access-s8h7r\") pod \"ingress-canary-vppxs\" (UID: \"faf3ad2e-6304-49c5-be6e-560a500240f8\") " pod="openshift-ingress-canary/ingress-canary-vppxs" Jul 08 19:42:33.767739 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.767600 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/faf3ad2e-6304-49c5-be6e-560a500240f8-cert\") pod \"ingress-canary-vppxs\" (UID: \"faf3ad2e-6304-49c5-be6e-560a500240f8\") " pod="openshift-ingress-canary/ingress-canary-vppxs" Jul 08 19:42:33.825902 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.825869 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:42:33.826085 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.826025 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:42:33.829193 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.829176 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dpn4j\"" Jul 08 19:42:33.829323 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.829175 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 19:42:33.829323 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.829175 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 19:42:33.868457 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.868428 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba1dead2-f97c-4a89-b5d1-487641ab971a-tmp-dir\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:42:33.868606 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.868476 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:42:33.868606 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.868531 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5ghn\" (UniqueName: \"kubernetes.io/projected/ba1dead2-f97c-4a89-b5d1-487641ab971a-kube-api-access-m5ghn\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:42:33.868606 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.868587 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8h7r\" (UniqueName: \"kubernetes.io/projected/faf3ad2e-6304-49c5-be6e-560a500240f8-kube-api-access-s8h7r\") pod \"ingress-canary-vppxs\" (UID: \"faf3ad2e-6304-49c5-be6e-560a500240f8\") " pod="openshift-ingress-canary/ingress-canary-vppxs" Jul 08 19:42:33.868606 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.868605 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ba1dead2-f97c-4a89-b5d1-487641ab971a-config-volume\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:42:33.868736 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.868673 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/faf3ad2e-6304-49c5-be6e-560a500240f8-cert\") pod \"ingress-canary-vppxs\" (UID: \"faf3ad2e-6304-49c5-be6e-560a500240f8\") " pod="openshift-ingress-canary/ingress-canary-vppxs" Jul 08 19:42:33.873020 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.872911 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/faf3ad2e-6304-49c5-be6e-560a500240f8-cert\") pod \"ingress-canary-vppxs\" (UID: \"faf3ad2e-6304-49c5-be6e-560a500240f8\") " pod="openshift-ingress-canary/ingress-canary-vppxs" Jul 08 19:42:33.877284 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.877264 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8h7r\" (UniqueName: \"kubernetes.io/projected/faf3ad2e-6304-49c5-be6e-560a500240f8-kube-api-access-s8h7r\") pod \"ingress-canary-vppxs\" (UID: \"faf3ad2e-6304-49c5-be6e-560a500240f8\") " pod="openshift-ingress-canary/ingress-canary-vppxs" Jul 08 19:42:33.969480 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.969390 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ba1dead2-f97c-4a89-b5d1-487641ab971a-config-volume\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:42:33.969627 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.969547 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba1dead2-f97c-4a89-b5d1-487641ab971a-tmp-dir\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:42:33.969627 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.969619 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:42:33.969709 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.969645 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5ghn\" (UniqueName: \"kubernetes.io/projected/ba1dead2-f97c-4a89-b5d1-487641ab971a-kube-api-access-m5ghn\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:42:33.969749 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:33.969727 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:42:33.969811 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:33.969800 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls podName:ba1dead2-f97c-4a89-b5d1-487641ab971a nodeName:}" failed. No retries permitted until 2026-07-08 19:42:34.46978296 +0000 UTC m=+36.163092437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls") pod "dns-default-bvr92" (UID: "ba1dead2-f97c-4a89-b5d1-487641ab971a") : secret "dns-default-metrics-tls" not found Jul 08 19:42:33.969872 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.969854 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba1dead2-f97c-4a89-b5d1-487641ab971a-tmp-dir\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:42:33.969959 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.969944 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ba1dead2-f97c-4a89-b5d1-487641ab971a-config-volume\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:42:33.980824 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.980799 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5ghn\" (UniqueName: \"kubernetes.io/projected/ba1dead2-f97c-4a89-b5d1-487641ab971a-kube-api-access-m5ghn\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:42:33.982424 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.982399 2568 generic.go:356] "Generic (PLEG): container finished" podID="d277414e-55a2-411f-852d-aae01918e1c4" containerID="0344de300975cf84b54a619a5046aef06e6199cd24f3eeaed3325dcf9d7a151d" exitCode=0 Jul 08 19:42:33.982544 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:33.982456 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g5gnj" event={"ID":"d277414e-55a2-411f-852d-aae01918e1c4","Type":"ContainerDied","Data":"0344de300975cf84b54a619a5046aef06e6199cd24f3eeaed3325dcf9d7a151d"} Jul 08 19:42:34.001964 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:34.001941 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vppxs" Jul 08 19:42:34.197495 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:34.197466 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vppxs"] Jul 08 19:42:34.200955 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:42:34.200925 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaf3ad2e_6304_49c5_be6e_560a500240f8.slice/crio-913e496f8f6d914f97feceeace19c39caf18b803f004f5f25c194cda64bca420 WatchSource:0}: Error finding container 913e496f8f6d914f97feceeace19c39caf18b803f004f5f25c194cda64bca420: Status 404 returned error can't find the container with id 913e496f8f6d914f97feceeace19c39caf18b803f004f5f25c194cda64bca420 Jul 08 19:42:34.473999 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:34.473964 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:42:34.474183 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:34.474087 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:42:34.474183 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:34.474139 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls podName:ba1dead2-f97c-4a89-b5d1-487641ab971a nodeName:}" failed. No retries permitted until 2026-07-08 19:42:35.474125318 +0000 UTC m=+37.167434782 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls") pod "dns-default-bvr92" (UID: "ba1dead2-f97c-4a89-b5d1-487641ab971a") : secret "dns-default-metrics-tls" not found Jul 08 19:42:34.826051 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:34.826006 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:42:34.829326 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:34.829303 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 19:42:34.830668 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:34.830645 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 19:42:34.830784 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:34.830681 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-84p6s\"" Jul 08 19:42:34.986198 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:34.986160 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vppxs" event={"ID":"faf3ad2e-6304-49c5-be6e-560a500240f8","Type":"ContainerStarted","Data":"913e496f8f6d914f97feceeace19c39caf18b803f004f5f25c194cda64bca420"} Jul 08 19:42:35.482729 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:35.482687 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:42:35.483182 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:35.482870 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:42:35.483182 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:35.482946 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls podName:ba1dead2-f97c-4a89-b5d1-487641ab971a nodeName:}" failed. No retries permitted until 2026-07-08 19:42:37.482920834 +0000 UTC m=+39.176230302 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls") pod "dns-default-bvr92" (UID: "ba1dead2-f97c-4a89-b5d1-487641ab971a") : secret "dns-default-metrics-tls" not found Jul 08 19:42:36.127625 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:36.127583 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:37.497908 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:37.497653 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:42:37.498513 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:37.497842 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:42:37.498513 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:37.498052 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls podName:ba1dead2-f97c-4a89-b5d1-487641ab971a nodeName:}" failed. No retries permitted until 2026-07-08 19:42:41.498009934 +0000 UTC m=+43.191319403 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls") pod "dns-default-bvr92" (UID: "ba1dead2-f97c-4a89-b5d1-487641ab971a") : secret "dns-default-metrics-tls" not found Jul 08 19:42:37.994521 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:37.994471 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vppxs" event={"ID":"faf3ad2e-6304-49c5-be6e-560a500240f8","Type":"ContainerStarted","Data":"76a0fddb1676165128ac51eafc06fb2d7277953f4c9d534c9a7fabdcbbbe6152"} Jul 08 19:42:38.011384 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:38.011334 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vppxs" podStartSLOduration=2.129566023 podStartE2EDuration="5.0113186s" totalImagesPullingTime="2.881752577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:38.010576889 +0000 UTC m=+39.703886377" watchObservedRunningTime="2026-07-08 19:42:38.0113186 +0000 UTC m=+39.704628086" Jul 08 19:42:38.892212 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:38.892148 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:38.947859 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:38.947825 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:40.003460 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:40.003429 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:40.056496 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:40.056463 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:41.005506 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:41.005469 2568 generic.go:356] "Generic (PLEG): container finished" podID="d277414e-55a2-411f-852d-aae01918e1c4" containerID="88e71c7627a4a507c592d3020b04d08ee0e7b347b01dd716813d657d77a56a90" exitCode=0 Jul 08 19:42:41.005506 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:41.005511 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g5gnj" event={"ID":"d277414e-55a2-411f-852d-aae01918e1c4","Type":"ContainerDied","Data":"88e71c7627a4a507c592d3020b04d08ee0e7b347b01dd716813d657d77a56a90"} Jul 08 19:42:41.528175 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:41.528142 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:42:41.528314 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:41.528264 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:42:41.528371 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:41.528314 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls podName:ba1dead2-f97c-4a89-b5d1-487641ab971a nodeName:}" failed. No retries permitted until 2026-07-08 19:42:49.528301435 +0000 UTC m=+51.221610899 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls") pod "dns-default-bvr92" (UID: "ba1dead2-f97c-4a89-b5d1-487641ab971a") : secret "dns-default-metrics-tls" not found Jul 08 19:42:42.011205 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:42.011173 2568 generic.go:356] "Generic (PLEG): container finished" podID="d277414e-55a2-411f-852d-aae01918e1c4" containerID="68836da0f119102fc94e2c513961c9fc33184cd22647c314c5f5a7d0827fd223" exitCode=0 Jul 08 19:42:42.011587 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:42.011217 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g5gnj" event={"ID":"d277414e-55a2-411f-852d-aae01918e1c4","Type":"ContainerDied","Data":"68836da0f119102fc94e2c513961c9fc33184cd22647c314c5f5a7d0827fd223"} Jul 08 19:42:43.016979 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:43.016940 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g5gnj" event={"ID":"d277414e-55a2-411f-852d-aae01918e1c4","Type":"ContainerStarted","Data":"3e10d1ce547b9d462b9267637e0b95073b2f59787c36459bd7ba6ccc5c2a85a9"} Jul 08 19:42:43.042302 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:43.042247 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-g5gnj" podStartSLOduration=9.057395213 podStartE2EDuration="44.042233376s" totalImagesPullingTime="34.984838163s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:43.041899886 +0000 UTC m=+44.735209373" watchObservedRunningTime="2026-07-08 19:42:43.042233376 +0000 UTC m=+44.735542861" Jul 08 19:42:43.161715 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:43.161681 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:43.224655 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:43.224623 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:43.776464 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:43.776426 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:43.876496 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:43.876454 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:47.349626 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:47.349588 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:48.164375 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:48.164336 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:48.264141 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:48.264100 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:49.303652 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:49.303613 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:49.589925 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:49.589897 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:42:49.590147 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:49.590057 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:42:49.590147 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:42:49.590111 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls podName:ba1dead2-f97c-4a89-b5d1-487641ab971a nodeName:}" failed. No retries permitted until 2026-07-08 19:43:05.590097307 +0000 UTC m=+67.283406772 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls") pod "dns-default-bvr92" (UID: "ba1dead2-f97c-4a89-b5d1-487641ab971a") : secret "dns-default-metrics-tls" not found Jul 08 19:42:49.626558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:49.626526 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:52.246426 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:52.246388 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:52.308224 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:52.308181 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:57.972787 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:57.972758 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lsfpc" Jul 08 19:42:58.140434 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:58.140395 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:58.201026 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:58.200989 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:42:58.253194 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:42:58.253100 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:04.494259 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:04.494212 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret\") pod \"global-pull-secret-syncer-82lzg\" (UID: \"3184c318-9df8-4f30-92e9-5bf8786149e9\") " pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:43:04.497380 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:04.497360 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 19:43:04.507954 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:04.507931 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3184c318-9df8-4f30-92e9-5bf8786149e9-original-pull-secret\") pod \"global-pull-secret-syncer-82lzg\" (UID: \"3184c318-9df8-4f30-92e9-5bf8786149e9\") " pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:43:04.594604 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:04.594566 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs\") pod \"network-metrics-daemon-s7hbd\" (UID: \"3b5311a0-d5fa-4994-9071-d861d2a53b09\") " pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:43:04.594800 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:04.594612 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89clm\" (UniqueName: \"kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm\") pod \"network-check-target-q6qvd\" (UID: \"dc75384f-fbaa-4ddf-b258-02a4024686aa\") " pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:43:04.597566 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:04.597546 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 19:43:04.597644 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:04.597609 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 19:43:04.605071 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:43:04.605030 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 19:43:04.605184 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:43:04.605115 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs podName:3b5311a0-d5fa-4994-9071-d861d2a53b09 nodeName:}" failed. No retries permitted until 2026-07-08 19:44:08.605099098 +0000 UTC m=+130.298408565 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs") pod "network-metrics-daemon-s7hbd" (UID: "3b5311a0-d5fa-4994-9071-d861d2a53b09") : secret "metrics-daemon-secret" not found Jul 08 19:43:04.607559 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:04.607538 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 19:43:04.618317 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:04.618293 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89clm\" (UniqueName: \"kubernetes.io/projected/dc75384f-fbaa-4ddf-b258-02a4024686aa-kube-api-access-89clm\") pod \"network-check-target-q6qvd\" (UID: \"dc75384f-fbaa-4ddf-b258-02a4024686aa\") " pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:43:04.734817 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:04.734770 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82lzg" Jul 08 19:43:04.840826 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:04.840801 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-84p6s\"" Jul 08 19:43:04.848251 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:04.848228 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:43:04.853411 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:04.853385 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-82lzg"] Jul 08 19:43:04.856808 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:43:04.856781 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3184c318_9df8_4f30_92e9_5bf8786149e9.slice/crio-471a09257d2788ffea00f39ea30160591e2988347a44ebb56d3eb8c1c0beb660 WatchSource:0}: Error finding container 471a09257d2788ffea00f39ea30160591e2988347a44ebb56d3eb8c1c0beb660: Status 404 returned error can't find the container with id 471a09257d2788ffea00f39ea30160591e2988347a44ebb56d3eb8c1c0beb660 Jul 08 19:43:04.979005 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:04.978976 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q6qvd"] Jul 08 19:43:04.982353 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:43:04.982329 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc75384f_fbaa_4ddf_b258_02a4024686aa.slice/crio-ea09250c7e4fb2b016c5933889236b3e540a05362bbd4b986463e46f5f4fa0ce WatchSource:0}: Error finding container ea09250c7e4fb2b016c5933889236b3e540a05362bbd4b986463e46f5f4fa0ce: Status 404 returned error can't find the container with id ea09250c7e4fb2b016c5933889236b3e540a05362bbd4b986463e46f5f4fa0ce Jul 08 19:43:05.068430 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:05.068397 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q6qvd" event={"ID":"dc75384f-fbaa-4ddf-b258-02a4024686aa","Type":"ContainerStarted","Data":"ea09250c7e4fb2b016c5933889236b3e540a05362bbd4b986463e46f5f4fa0ce"} Jul 08 19:43:05.069482 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:05.069460 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-82lzg" event={"ID":"3184c318-9df8-4f30-92e9-5bf8786149e9","Type":"ContainerStarted","Data":"471a09257d2788ffea00f39ea30160591e2988347a44ebb56d3eb8c1c0beb660"} Jul 08 19:43:05.603378 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:05.603344 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:43:05.603851 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:43:05.603517 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:43:05.603851 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:43:05.603598 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls podName:ba1dead2-f97c-4a89-b5d1-487641ab971a nodeName:}" failed. No retries permitted until 2026-07-08 19:43:37.603575947 +0000 UTC m=+99.296885416 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls") pod "dns-default-bvr92" (UID: "ba1dead2-f97c-4a89-b5d1-487641ab971a") : secret "dns-default-metrics-tls" not found Jul 08 19:43:08.173078 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:08.173009 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:08.273399 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:08.273347 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:08.329583 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:08.329541 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:10.084716 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:10.084683 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q6qvd" event={"ID":"dc75384f-fbaa-4ddf-b258-02a4024686aa","Type":"ContainerStarted","Data":"5202fe38f9264efa14401be5b1a9031d9f4e433835a519fee8c034f4adf20d91"} Jul 08 19:43:10.085183 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:10.084806 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:43:10.086173 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:10.086151 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-82lzg" event={"ID":"3184c318-9df8-4f30-92e9-5bf8786149e9","Type":"ContainerStarted","Data":"97d5199fe36c2c15bf9b57d08b5ba4570b89180fea131754fa26a289a7003269"} Jul 08 19:43:10.100104 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:10.100063 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-q6qvd" podStartSLOduration=66.30386301 podStartE2EDuration="1m11.100028809s" totalImagesPullingTime="4.796165799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:43:10.099644714 +0000 UTC m=+71.792954210" watchObservedRunningTime="2026-07-08 19:43:10.100028809 +0000 UTC m=+71.793338330" Jul 08 19:43:10.114383 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:10.114340 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-82lzg" podStartSLOduration=66.188188614 podStartE2EDuration="1m11.114326175s" totalImagesPullingTime="4.926137561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:43:10.113870037 +0000 UTC m=+71.807179525" watchObservedRunningTime="2026-07-08 19:43:10.114326175 +0000 UTC m=+71.807635660" Jul 08 19:43:37.611080 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:37.611015 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:43:37.611600 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:43:37.611191 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:43:37.611600 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:43:37.611322 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls podName:ba1dead2-f97c-4a89-b5d1-487641ab971a nodeName:}" failed. No retries permitted until 2026-07-08 19:44:41.611298436 +0000 UTC m=+163.304607922 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls") pod "dns-default-bvr92" (UID: "ba1dead2-f97c-4a89-b5d1-487641ab971a") : secret "dns-default-metrics-tls" not found Jul 08 19:43:41.091718 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:43:41.091691 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-q6qvd" Jul 08 19:44:08.619714 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:44:08.619664 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs\") pod \"network-metrics-daemon-s7hbd\" (UID: \"3b5311a0-d5fa-4994-9071-d861d2a53b09\") " pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:44:08.620215 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:44:08.619811 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 19:44:08.620215 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:44:08.619906 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs podName:3b5311a0-d5fa-4994-9071-d861d2a53b09 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:10.619886754 +0000 UTC m=+252.313196252 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs") pod "network-metrics-daemon-s7hbd" (UID: "3b5311a0-d5fa-4994-9071-d861d2a53b09") : secret "metrics-daemon-secret" not found Jul 08 19:44:36.725361 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:44:36.725312 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-bvr92" podUID="ba1dead2-f97c-4a89-b5d1-487641ab971a" Jul 08 19:44:36.840235 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:44:36.840197 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:44:41.643774 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:44:41.643708 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:44:41.644219 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:44:41.643859 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:44:41.644219 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:44:41.643938 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls podName:ba1dead2-f97c-4a89-b5d1-487641ab971a nodeName:}" failed. No retries permitted until 2026-07-08 19:46:43.64392085 +0000 UTC m=+285.337230313 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls") pod "dns-default-bvr92" (UID: "ba1dead2-f97c-4a89-b5d1-487641ab971a") : secret "dns-default-metrics-tls" not found Jul 08 19:44:48.827468 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:44:48.827426 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:44:49.826375 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:44:49.826317 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bvr92" Jul 08 19:45:26.598178 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:45:26.598149 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:05.251952 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.251913 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-45nzs"] Jul 08 19:46:05.253897 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.253875 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-45nzs" Jul 08 19:46:05.256708 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.256689 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-vntlt\"" Jul 08 19:46:05.275602 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.275576 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-45nzs"] Jul 08 19:46:05.303495 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.303461 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlcnw\" (UniqueName: \"kubernetes.io/projected/915073b7-e0c7-4dd7-9c86-c9dac143930f-kube-api-access-xlcnw\") pod \"network-check-source-989f8b8dc-45nzs\" (UID: \"915073b7-e0c7-4dd7-9c86-c9dac143930f\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-45nzs" Jul 08 19:46:05.348434 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.348398 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6vp7d"] Jul 08 19:46:05.350303 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.350283 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6vp7d" Jul 08 19:46:05.353349 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.353329 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 08 19:46:05.353460 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.353387 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-2p8t5\"" Jul 08 19:46:05.353460 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.353398 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:46:05.355969 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.355944 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66"] Jul 08 19:46:05.357718 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.357699 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" Jul 08 19:46:05.360697 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.360676 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 08 19:46:05.360966 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.360938 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 19:46:05.361113 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.361090 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 19:46:05.363114 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.363087 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 08 19:46:05.364669 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.364589 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-7swjf\"" Jul 08 19:46:05.365706 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.365679 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6vp7d"] Jul 08 19:46:05.370888 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.370870 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66"] Jul 08 19:46:05.404090 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.404059 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlcnw\" (UniqueName: \"kubernetes.io/projected/915073b7-e0c7-4dd7-9c86-c9dac143930f-kube-api-access-xlcnw\") pod \"network-check-source-989f8b8dc-45nzs\" (UID: \"915073b7-e0c7-4dd7-9c86-c9dac143930f\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-45nzs" Jul 08 19:46:05.404256 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.404106 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-6mr66\" (UID: \"c3fcc756-d506-4c6d-b7fc-39cf568fc9e2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" Jul 08 19:46:05.404256 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.404142 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn97k\" (UniqueName: \"kubernetes.io/projected/3c0e75a4-b434-4916-894b-19664ea0fd9c-kube-api-access-vn97k\") pod \"volume-data-source-validator-5d456fdbcd-6vp7d\" (UID: \"3c0e75a4-b434-4916-894b-19664ea0fd9c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6vp7d" Jul 08 19:46:05.404256 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.404168 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wgtt\" (UniqueName: \"kubernetes.io/projected/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-kube-api-access-9wgtt\") pod \"cluster-monitoring-operator-64786d9cbb-6mr66\" (UID: \"c3fcc756-d506-4c6d-b7fc-39cf568fc9e2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" Jul 08 19:46:05.404256 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.404202 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-6mr66\" (UID: \"c3fcc756-d506-4c6d-b7fc-39cf568fc9e2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" Jul 08 19:46:05.412886 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.412862 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlcnw\" (UniqueName: \"kubernetes.io/projected/915073b7-e0c7-4dd7-9c86-c9dac143930f-kube-api-access-xlcnw\") pod \"network-check-source-989f8b8dc-45nzs\" (UID: \"915073b7-e0c7-4dd7-9c86-c9dac143930f\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-45nzs" Jul 08 19:46:05.454686 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.454647 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d9fc557f9-5kxlf"] Jul 08 19:46:05.457507 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.457487 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.460453 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.460424 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 19:46:05.460453 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.460426 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 19:46:05.460777 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.460757 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 19:46:05.461102 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.461086 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4fdwr\"" Jul 08 19:46:05.466374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.466356 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 19:46:05.473415 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.473393 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d9fc557f9-5kxlf"] Jul 08 19:46:05.505368 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.505276 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wgtt\" (UniqueName: \"kubernetes.io/projected/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-kube-api-access-9wgtt\") pod \"cluster-monitoring-operator-64786d9cbb-6mr66\" (UID: \"c3fcc756-d506-4c6d-b7fc-39cf568fc9e2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" Jul 08 19:46:05.505368 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.505313 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ac68b53-fb73-41f4-9c2f-3a210910c168-installation-pull-secrets\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.505368 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.505337 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ac68b53-fb73-41f4-9c2f-3a210910c168-ca-trust-extracted\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.505368 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.505358 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-6mr66\" (UID: \"c3fcc756-d506-4c6d-b7fc-39cf568fc9e2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" Jul 08 19:46:05.505693 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.505375 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ac68b53-fb73-41f4-9c2f-3a210910c168-image-registry-private-configuration\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.505693 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.505392 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ac68b53-fb73-41f4-9c2f-3a210910c168-trusted-ca\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.505693 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.505470 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-bound-sa-token\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.505693 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.505554 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.505693 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.505586 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-certificates\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.505693 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.505645 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-6mr66\" (UID: \"c3fcc756-d506-4c6d-b7fc-39cf568fc9e2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" Jul 08 19:46:05.505693 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.505671 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsz4v\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-kube-api-access-wsz4v\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.505975 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.505708 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn97k\" (UniqueName: \"kubernetes.io/projected/3c0e75a4-b434-4916-894b-19664ea0fd9c-kube-api-access-vn97k\") pod \"volume-data-source-validator-5d456fdbcd-6vp7d\" (UID: \"3c0e75a4-b434-4916-894b-19664ea0fd9c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6vp7d" Jul 08 19:46:05.505975 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:05.505731 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 19:46:05.505975 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:05.505798 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls podName:c3fcc756-d506-4c6d-b7fc-39cf568fc9e2 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:06.005778046 +0000 UTC m=+247.699087510 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-6mr66" (UID: "c3fcc756-d506-4c6d-b7fc-39cf568fc9e2") : secret "cluster-monitoring-operator-tls" not found Jul 08 19:46:05.506107 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.506085 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-6mr66\" (UID: \"c3fcc756-d506-4c6d-b7fc-39cf568fc9e2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" Jul 08 19:46:05.516407 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.516376 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn97k\" (UniqueName: \"kubernetes.io/projected/3c0e75a4-b434-4916-894b-19664ea0fd9c-kube-api-access-vn97k\") pod \"volume-data-source-validator-5d456fdbcd-6vp7d\" (UID: \"3c0e75a4-b434-4916-894b-19664ea0fd9c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6vp7d" Jul 08 19:46:05.516513 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.516411 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wgtt\" (UniqueName: \"kubernetes.io/projected/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-kube-api-access-9wgtt\") pod \"cluster-monitoring-operator-64786d9cbb-6mr66\" (UID: \"c3fcc756-d506-4c6d-b7fc-39cf568fc9e2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" Jul 08 19:46:05.562274 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.562237 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-45nzs" Jul 08 19:46:05.590175 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.590126 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:05.606107 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.606076 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsz4v\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-kube-api-access-wsz4v\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.606268 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.606131 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ac68b53-fb73-41f4-9c2f-3a210910c168-installation-pull-secrets\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.606268 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.606163 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ac68b53-fb73-41f4-9c2f-3a210910c168-ca-trust-extracted\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.606268 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.606193 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ac68b53-fb73-41f4-9c2f-3a210910c168-image-registry-private-configuration\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.606268 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.606217 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ac68b53-fb73-41f4-9c2f-3a210910c168-trusted-ca\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.606268 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.606251 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-bound-sa-token\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.606960 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.606307 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.606960 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.606360 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-certificates\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.606960 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.606604 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ac68b53-fb73-41f4-9c2f-3a210910c168-ca-trust-extracted\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.606960 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.606942 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-certificates\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.607183 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:05.606966 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:46:05.607183 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:05.606985 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d9fc557f9-5kxlf: secret "image-registry-tls" not found Jul 08 19:46:05.607183 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:05.607056 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls podName:9ac68b53-fb73-41f4-9c2f-3a210910c168 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:06.107018411 +0000 UTC m=+247.800327900 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls") pod "image-registry-7d9fc557f9-5kxlf" (UID: "9ac68b53-fb73-41f4-9c2f-3a210910c168") : secret "image-registry-tls" not found Jul 08 19:46:05.608493 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.607391 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ac68b53-fb73-41f4-9c2f-3a210910c168-trusted-ca\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.610152 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.610125 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ac68b53-fb73-41f4-9c2f-3a210910c168-image-registry-private-configuration\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.610236 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.610191 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ac68b53-fb73-41f4-9c2f-3a210910c168-installation-pull-secrets\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.615356 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.615325 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-bound-sa-token\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.617488 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.617466 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsz4v\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-kube-api-access-wsz4v\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:05.659340 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.659309 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6vp7d" Jul 08 19:46:05.681028 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.680993 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-45nzs"] Jul 08 19:46:05.684017 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:46:05.683987 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod915073b7_e0c7_4dd7_9c86_c9dac143930f.slice/crio-c29a9f4dacbc3f3ff676d41a21d6585cd98c2211298c13a63f8e5bec7734fbf3 WatchSource:0}: Error finding container c29a9f4dacbc3f3ff676d41a21d6585cd98c2211298c13a63f8e5bec7734fbf3: Status 404 returned error can't find the container with id c29a9f4dacbc3f3ff676d41a21d6585cd98c2211298c13a63f8e5bec7734fbf3 Jul 08 19:46:05.684172 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.684153 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:05.784584 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:05.784501 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6vp7d"] Jul 08 19:46:05.787730 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:46:05.787695 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c0e75a4_b434_4916_894b_19664ea0fd9c.slice/crio-3c376d53292460cf59b1a1bd0b8505da8fa7e1a7d4271fd0c0818998f468685f WatchSource:0}: Error finding container 3c376d53292460cf59b1a1bd0b8505da8fa7e1a7d4271fd0c0818998f468685f: Status 404 returned error can't find the container with id 3c376d53292460cf59b1a1bd0b8505da8fa7e1a7d4271fd0c0818998f468685f Jul 08 19:46:06.010013 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:06.009978 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-6mr66\" (UID: \"c3fcc756-d506-4c6d-b7fc-39cf568fc9e2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" Jul 08 19:46:06.010214 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:06.010151 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 19:46:06.010268 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:06.010222 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls podName:c3fcc756-d506-4c6d-b7fc-39cf568fc9e2 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:07.010205338 +0000 UTC m=+248.703514802 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-6mr66" (UID: "c3fcc756-d506-4c6d-b7fc-39cf568fc9e2") : secret "cluster-monitoring-operator-tls" not found Jul 08 19:46:06.111106 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:06.111069 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:06.111307 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:06.111235 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:46:06.111307 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:06.111262 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d9fc557f9-5kxlf: secret "image-registry-tls" not found Jul 08 19:46:06.111418 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:06.111330 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls podName:9ac68b53-fb73-41f4-9c2f-3a210910c168 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:07.111309383 +0000 UTC m=+248.804618851 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls") pod "image-registry-7d9fc557f9-5kxlf" (UID: "9ac68b53-fb73-41f4-9c2f-3a210910c168") : secret "image-registry-tls" not found Jul 08 19:46:06.499796 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:06.499709 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6vp7d" event={"ID":"3c0e75a4-b434-4916-894b-19664ea0fd9c","Type":"ContainerStarted","Data":"3c376d53292460cf59b1a1bd0b8505da8fa7e1a7d4271fd0c0818998f468685f"} Jul 08 19:46:06.501502 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:06.501475 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-45nzs" event={"ID":"915073b7-e0c7-4dd7-9c86-c9dac143930f","Type":"ContainerStarted","Data":"4865576f540805f4f448b32607f34d77cc302b644411585464637cd01473225a"} Jul 08 19:46:06.501633 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:06.501514 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-45nzs" event={"ID":"915073b7-e0c7-4dd7-9c86-c9dac143930f","Type":"ContainerStarted","Data":"c29a9f4dacbc3f3ff676d41a21d6585cd98c2211298c13a63f8e5bec7734fbf3"} Jul 08 19:46:06.518182 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:06.518125 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-45nzs" podStartSLOduration=1.518109586 podStartE2EDuration="1.518109586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:46:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:46:06.517752752 +0000 UTC m=+248.211062240" watchObservedRunningTime="2026-07-08 19:46:06.518109586 +0000 UTC m=+248.211419093" Jul 08 19:46:07.019092 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:07.019058 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-6mr66\" (UID: \"c3fcc756-d506-4c6d-b7fc-39cf568fc9e2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" Jul 08 19:46:07.019236 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:07.019208 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 19:46:07.019288 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:07.019274 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls podName:c3fcc756-d506-4c6d-b7fc-39cf568fc9e2 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:09.019257216 +0000 UTC m=+250.712566685 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-6mr66" (UID: "c3fcc756-d506-4c6d-b7fc-39cf568fc9e2") : secret "cluster-monitoring-operator-tls" not found Jul 08 19:46:07.119659 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:07.119614 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:07.119813 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:07.119769 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:46:07.119813 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:07.119790 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d9fc557f9-5kxlf: secret "image-registry-tls" not found Jul 08 19:46:07.120024 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:07.119848 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls podName:9ac68b53-fb73-41f4-9c2f-3a210910c168 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:09.119832879 +0000 UTC m=+250.813142348 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls") pod "image-registry-7d9fc557f9-5kxlf" (UID: "9ac68b53-fb73-41f4-9c2f-3a210910c168") : secret "image-registry-tls" not found Jul 08 19:46:07.506605 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:07.506567 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6vp7d" event={"ID":"3c0e75a4-b434-4916-894b-19664ea0fd9c","Type":"ContainerStarted","Data":"a54c7d8b22602f0f3d3c41aa290c2dba93cf61f0c03a0f27de6d8255992fa616"} Jul 08 19:46:07.525521 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:07.525469 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6vp7d" podStartSLOduration=1.279260383 podStartE2EDuration="2.525455765s" totalImagesPullingTime="1.246195382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:46:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:46:07.523504106 +0000 UTC m=+249.216813592" watchObservedRunningTime="2026-07-08 19:46:07.525455765 +0000 UTC m=+249.218765251" Jul 08 19:46:07.604158 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:07.604126 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:07.661447 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:07.661416 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:09.032383 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:09.032345 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-6mr66\" (UID: \"c3fcc756-d506-4c6d-b7fc-39cf568fc9e2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" Jul 08 19:46:09.032789 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:09.032489 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 19:46:09.032789 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:09.032557 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls podName:c3fcc756-d506-4c6d-b7fc-39cf568fc9e2 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:13.032541862 +0000 UTC m=+254.725851326 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-6mr66" (UID: "c3fcc756-d506-4c6d-b7fc-39cf568fc9e2") : secret "cluster-monitoring-operator-tls" not found Jul 08 19:46:09.133670 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:09.133633 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:09.133823 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:09.133756 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:46:09.133823 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:09.133768 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d9fc557f9-5kxlf: secret "image-registry-tls" not found Jul 08 19:46:09.133823 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:09.133814 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls podName:9ac68b53-fb73-41f4-9c2f-3a210910c168 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:13.133800433 +0000 UTC m=+254.827109910 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls") pod "image-registry-7d9fc557f9-5kxlf" (UID: "9ac68b53-fb73-41f4-9c2f-3a210910c168") : secret "image-registry-tls" not found Jul 08 19:46:10.600854 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.600821 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:10.643283 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.643242 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs\") pod \"network-metrics-daemon-s7hbd\" (UID: \"3b5311a0-d5fa-4994-9071-d861d2a53b09\") " pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:46:10.643472 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:10.643416 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 19:46:10.643540 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:10.643507 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs podName:3b5311a0-d5fa-4994-9071-d861d2a53b09 nodeName:}" failed. No retries permitted until 2026-07-08 19:48:12.643486504 +0000 UTC m=+374.336795968 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs") pod "network-metrics-daemon-s7hbd" (UID: "3b5311a0-d5fa-4994-9071-d861d2a53b09") : secret "metrics-daemon-secret" not found Jul 08 19:46:10.670709 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.670680 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:10.710090 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.710055 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6qk8p"] Jul 08 19:46:10.712215 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.712193 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:10.714946 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.714921 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 19:46:10.715196 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.715180 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q8lf2\"" Jul 08 19:46:10.716466 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.716447 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 19:46:10.716564 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.716449 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 19:46:10.716615 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.716602 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 19:46:10.724147 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.724121 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6qk8p"] Jul 08 19:46:10.743876 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.743837 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzqwx\" (UniqueName: \"kubernetes.io/projected/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-api-access-wzqwx\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:10.744150 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.743903 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a0479c7a-9963-4c7d-be2b-2860133eb4a4-crio-socket\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:10.744150 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.743945 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:10.744150 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.744059 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0479c7a-9963-4c7d-be2b-2860133eb4a4-data-volume\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:10.744150 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.744137 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a0479c7a-9963-4c7d-be2b-2860133eb4a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:10.744818 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.744789 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:10.836979 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.836922 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:10.845020 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.844992 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:10.845166 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.845093 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0479c7a-9963-4c7d-be2b-2860133eb4a4-data-volume\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:10.845166 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:10.845122 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:46:10.845166 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.845149 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a0479c7a-9963-4c7d-be2b-2860133eb4a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:10.845374 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:10.845194 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-rbac-proxy-cm podName:a0479c7a-9963-4c7d-be2b-2860133eb4a4 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:11.345177571 +0000 UTC m=+253.038487064 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6qk8p" (UID: "a0479c7a-9963-4c7d-be2b-2860133eb4a4") : configmap "kube-rbac-proxy" not found Jul 08 19:46:10.845374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.845223 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzqwx\" (UniqueName: \"kubernetes.io/projected/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-api-access-wzqwx\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:10.845374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.845256 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a0479c7a-9963-4c7d-be2b-2860133eb4a4-crio-socket\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:10.845374 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:10.845264 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:46:10.845374 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:10.845318 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0479c7a-9963-4c7d-be2b-2860133eb4a4-insights-runtime-extractor-tls podName:a0479c7a-9963-4c7d-be2b-2860133eb4a4 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:11.345300813 +0000 UTC m=+253.038610276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a0479c7a-9963-4c7d-be2b-2860133eb4a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6qk8p" (UID: "a0479c7a-9963-4c7d-be2b-2860133eb4a4") : secret "insights-runtime-extractor-tls" not found Jul 08 19:46:10.845374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.845322 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a0479c7a-9963-4c7d-be2b-2860133eb4a4-crio-socket\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:10.845569 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.845427 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0479c7a-9963-4c7d-be2b-2860133eb4a4-data-volume\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:10.854688 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.854632 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzqwx\" (UniqueName: \"kubernetes.io/projected/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-api-access-wzqwx\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:10.940535 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:10.940504 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:11.039662 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.039629 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:11.043028 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.043002 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-6n58q"] Jul 08 19:46:11.046742 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.046726 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6n58q" Jul 08 19:46:11.049598 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.049579 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 08 19:46:11.049679 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.049593 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-xs79x\"" Jul 08 19:46:11.049679 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.049587 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 08 19:46:11.056649 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.056630 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-6n58q"] Jul 08 19:46:11.139063 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.138931 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:11.146964 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.146943 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzllw\" (UniqueName: \"kubernetes.io/projected/d95cfbc4-4f29-4b28-b37e-7869239b6302-kube-api-access-gzllw\") pod \"migrator-d57d58cc9-6n58q\" (UID: \"d95cfbc4-4f29-4b28-b37e-7869239b6302\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6n58q" Jul 08 19:46:11.239606 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.239560 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:11.247675 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.247650 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzllw\" (UniqueName: \"kubernetes.io/projected/d95cfbc4-4f29-4b28-b37e-7869239b6302-kube-api-access-gzllw\") pod \"migrator-d57d58cc9-6n58q\" (UID: \"d95cfbc4-4f29-4b28-b37e-7869239b6302\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6n58q" Jul 08 19:46:11.256321 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.256292 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzllw\" (UniqueName: \"kubernetes.io/projected/d95cfbc4-4f29-4b28-b37e-7869239b6302-kube-api-access-gzllw\") pod \"migrator-d57d58cc9-6n58q\" (UID: \"d95cfbc4-4f29-4b28-b37e-7869239b6302\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6n58q" Jul 08 19:46:11.339588 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.339558 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:11.348712 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.348688 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a0479c7a-9963-4c7d-be2b-2860133eb4a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:11.348789 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.348743 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:11.348826 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:11.348810 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:46:11.348862 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:11.348824 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:46:11.348862 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:11.348860 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-rbac-proxy-cm podName:a0479c7a-9963-4c7d-be2b-2860133eb4a4 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:12.348846259 +0000 UTC m=+254.042155723 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6qk8p" (UID: "a0479c7a-9963-4c7d-be2b-2860133eb4a4") : configmap "kube-rbac-proxy" not found Jul 08 19:46:11.348931 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:11.348872 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0479c7a-9963-4c7d-be2b-2860133eb4a4-insights-runtime-extractor-tls podName:a0479c7a-9963-4c7d-be2b-2860133eb4a4 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:12.348866664 +0000 UTC m=+254.042176127 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a0479c7a-9963-4c7d-be2b-2860133eb4a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6qk8p" (UID: "a0479c7a-9963-4c7d-be2b-2860133eb4a4") : secret "insights-runtime-extractor-tls" not found Jul 08 19:46:11.355639 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.355615 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6n58q" Jul 08 19:46:11.439798 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.439769 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:11.476677 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.476634 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-6n58q"] Jul 08 19:46:11.479237 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:46:11.479211 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd95cfbc4_4f29_4b28_b37e_7869239b6302.slice/crio-dc974dafcc5991b37ff98368fa95050e30c422ecf80a3a326e39afe6b32bcef1 WatchSource:0}: Error finding container dc974dafcc5991b37ff98368fa95050e30c422ecf80a3a326e39afe6b32bcef1: Status 404 returned error can't find the container with id dc974dafcc5991b37ff98368fa95050e30c422ecf80a3a326e39afe6b32bcef1 Jul 08 19:46:11.518358 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:11.518322 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6n58q" event={"ID":"d95cfbc4-4f29-4b28-b37e-7869239b6302","Type":"ContainerStarted","Data":"dc974dafcc5991b37ff98368fa95050e30c422ecf80a3a326e39afe6b32bcef1"} Jul 08 19:46:12.357181 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:12.357142 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a0479c7a-9963-4c7d-be2b-2860133eb4a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:12.357596 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:12.357205 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:12.357596 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:12.357269 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:46:12.357596 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:12.357306 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:46:12.357596 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:12.357318 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-rbac-proxy-cm podName:a0479c7a-9963-4c7d-be2b-2860133eb4a4 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:14.35730424 +0000 UTC m=+256.050613704 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6qk8p" (UID: "a0479c7a-9963-4c7d-be2b-2860133eb4a4") : configmap "kube-rbac-proxy" not found Jul 08 19:46:12.357596 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:12.357413 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0479c7a-9963-4c7d-be2b-2860133eb4a4-insights-runtime-extractor-tls podName:a0479c7a-9963-4c7d-be2b-2860133eb4a4 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:14.357400437 +0000 UTC m=+256.050709902 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a0479c7a-9963-4c7d-be2b-2860133eb4a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6qk8p" (UID: "a0479c7a-9963-4c7d-be2b-2860133eb4a4") : secret "insights-runtime-extractor-tls" not found Jul 08 19:46:12.893612 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:12.893587 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rz46d_d69db7df-9f0a-4339-99b0-1d7dffa3e30e/dns-node-resolver/0.log" Jul 08 19:46:13.063157 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:13.063112 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-6mr66\" (UID: \"c3fcc756-d506-4c6d-b7fc-39cf568fc9e2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" Jul 08 19:46:13.063329 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:13.063256 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 19:46:13.063394 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:13.063329 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls podName:c3fcc756-d506-4c6d-b7fc-39cf568fc9e2 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:21.06331116 +0000 UTC m=+262.756620624 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-6mr66" (UID: "c3fcc756-d506-4c6d-b7fc-39cf568fc9e2") : secret "cluster-monitoring-operator-tls" not found Jul 08 19:46:13.095934 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:13.095899 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:13.159857 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:13.159779 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:13.163929 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:13.163905 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:13.164065 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:13.164051 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:46:13.164113 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:13.164066 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d9fc557f9-5kxlf: secret "image-registry-tls" not found Jul 08 19:46:13.164150 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:13.164113 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls podName:9ac68b53-fb73-41f4-9c2f-3a210910c168 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:21.164099721 +0000 UTC m=+262.857409214 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls") pod "image-registry-7d9fc557f9-5kxlf" (UID: "9ac68b53-fb73-41f4-9c2f-3a210910c168") : secret "image-registry-tls" not found Jul 08 19:46:13.493677 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:13.493595 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-62sgn_a5cf071e-9c63-46a7-8fef-f1f11d2a43cd/node-ca/0.log" Jul 08 19:46:13.852613 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:13.852586 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:14.374148 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:14.374098 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a0479c7a-9963-4c7d-be2b-2860133eb4a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:14.374331 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:14.374170 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:14.374331 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:14.374242 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:46:14.374331 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:14.374257 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:46:14.374331 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:14.374295 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-rbac-proxy-cm podName:a0479c7a-9963-4c7d-be2b-2860133eb4a4 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:18.374280638 +0000 UTC m=+260.067590101 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6qk8p" (UID: "a0479c7a-9963-4c7d-be2b-2860133eb4a4") : configmap "kube-rbac-proxy" not found Jul 08 19:46:14.374331 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:14.374315 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0479c7a-9963-4c7d-be2b-2860133eb4a4-insights-runtime-extractor-tls podName:a0479c7a-9963-4c7d-be2b-2860133eb4a4 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:18.374301091 +0000 UTC m=+260.067610555 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a0479c7a-9963-4c7d-be2b-2860133eb4a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6qk8p" (UID: "a0479c7a-9963-4c7d-be2b-2860133eb4a4") : secret "insights-runtime-extractor-tls" not found Jul 08 19:46:14.492319 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:14.492285 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:14.528172 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:14.528138 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6n58q" event={"ID":"d95cfbc4-4f29-4b28-b37e-7869239b6302","Type":"ContainerStarted","Data":"70132cb086875d01fb22efefd5a77cf4c3225dd146326e7f4f0fb9db63546332"} Jul 08 19:46:14.528629 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:14.528237 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6n58q" event={"ID":"d95cfbc4-4f29-4b28-b37e-7869239b6302","Type":"ContainerStarted","Data":"80175b917c59b216a106f761c0d5504b6cb861f17d8300371308bf220aec4dee"} Jul 08 19:46:14.547075 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:14.547011 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6n58q" podStartSLOduration=1.247916789 podStartE2EDuration="3.54699702s" totalImagesPullingTime="2.299080231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:46:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:46:14.544793034 +0000 UTC m=+256.238102521" watchObservedRunningTime="2026-07-08 19:46:14.54699702 +0000 UTC m=+256.240306506" Jul 08 19:46:14.604990 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:14.604959 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:14.696405 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:14.696334 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vppxs_faf3ad2e-6304-49c5-be6e-560a500240f8/serve-healthcheck-canary/0.log" Jul 08 19:46:14.708472 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:14.708446 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:14.843579 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:14.843547 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:14.903572 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:14.903537 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:15.206388 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:15.206356 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:16.190258 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:16.190220 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:16.588159 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:16.588125 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:16.786908 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:16.786876 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:16.991173 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:16.991077 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:17.192907 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:17.192875 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:17.363566 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:17.363532 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:18.033925 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:18.033896 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:18.386161 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:18.386133 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:18.407269 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:18.407240 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:18.407341 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:18.407311 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a0479c7a-9963-4c7d-be2b-2860133eb4a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:18.407373 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:18.407362 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:46:18.407425 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:18.407412 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:46:18.407425 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:18.407421 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-rbac-proxy-cm podName:a0479c7a-9963-4c7d-be2b-2860133eb4a4 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:26.407405919 +0000 UTC m=+268.100715382 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6qk8p" (UID: "a0479c7a-9963-4c7d-be2b-2860133eb4a4") : configmap "kube-rbac-proxy" not found Jul 08 19:46:18.407494 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:18.407455 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0479c7a-9963-4c7d-be2b-2860133eb4a4-insights-runtime-extractor-tls podName:a0479c7a-9963-4c7d-be2b-2860133eb4a4 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:26.407444239 +0000 UTC m=+268.100753703 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a0479c7a-9963-4c7d-be2b-2860133eb4a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6qk8p" (UID: "a0479c7a-9963-4c7d-be2b-2860133eb4a4") : secret "insights-runtime-extractor-tls" not found Jul 08 19:46:18.487252 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:18.487201 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:18.587269 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:18.587233 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:18.790483 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:18.790409 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:18.986237 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:18.986205 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:19.193837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:19.193805 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:19.987260 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:19.987231 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:20.586559 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:20.586514 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:20.787542 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:20.787512 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:20.893809 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:20.893726 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:20.995005 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:20.994971 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:21.130503 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:21.130450 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-6mr66\" (UID: \"c3fcc756-d506-4c6d-b7fc-39cf568fc9e2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" Jul 08 19:46:21.130685 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:21.130607 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 19:46:21.130685 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:21.130671 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls podName:c3fcc756-d506-4c6d-b7fc-39cf568fc9e2 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:37.130655921 +0000 UTC m=+278.823965385 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-6mr66" (UID: "c3fcc756-d506-4c6d-b7fc-39cf568fc9e2") : secret "cluster-monitoring-operator-tls" not found Jul 08 19:46:21.220532 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:21.220449 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:21.231716 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:21.231685 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:21.234133 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:21.234107 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls\") pod \"image-registry-7d9fc557f9-5kxlf\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:21.366095 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:21.366054 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:21.491847 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:21.491824 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d9fc557f9-5kxlf"] Jul 08 19:46:21.494062 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:46:21.494016 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac68b53_fb73_41f4_9c2f_3a210910c168.slice/crio-f4d51c2b7129eab329140c8db278072357ea605f786651f80292f57a6cc03ca3 WatchSource:0}: Error finding container f4d51c2b7129eab329140c8db278072357ea605f786651f80292f57a6cc03ca3: Status 404 returned error can't find the container with id f4d51c2b7129eab329140c8db278072357ea605f786651f80292f57a6cc03ca3 Jul 08 19:46:21.549669 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:21.549640 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" event={"ID":"9ac68b53-fb73-41f4-9c2f-3a210910c168","Type":"ContainerStarted","Data":"f4d51c2b7129eab329140c8db278072357ea605f786651f80292f57a6cc03ca3"} Jul 08 19:46:21.802902 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:21.802819 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:21.906776 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:21.906736 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:22.004732 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:22.004694 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:22.557482 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:22.557447 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" event={"ID":"9ac68b53-fb73-41f4-9c2f-3a210910c168","Type":"ContainerStarted","Data":"f7050887514ecf7020698987aea49c14b0b5ec16bf7b600b4dffb6e3a39e2d25"} Jul 08 19:46:22.557631 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:22.557558 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:22.579579 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:22.579528 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" podStartSLOduration=17.579511494 podStartE2EDuration="17.579511494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:46:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:46:22.577892514 +0000 UTC m=+264.271202000" watchObservedRunningTime="2026-07-08 19:46:22.579511494 +0000 UTC m=+264.272820980" Jul 08 19:46:22.589489 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:22.589456 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:22.787473 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:22.787421 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:23.037858 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:23.037821 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:23.400312 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:23.400277 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:23.935932 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:23.935894 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:24.198014 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:24.197929 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:24.392177 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:24.392146 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:24.582931 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:24.582900 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:24.785242 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:24.785211 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:25.188619 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:25.188588 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:25.764691 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:25.764658 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:26.474822 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:26.474788 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a0479c7a-9963-4c7d-be2b-2860133eb4a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:26.474999 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:26.474850 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:26.475352 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:26.475328 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a0479c7a-9963-4c7d-be2b-2860133eb4a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:26.477160 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:26.477138 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a0479c7a-9963-4c7d-be2b-2860133eb4a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6qk8p\" (UID: \"a0479c7a-9963-4c7d-be2b-2860133eb4a4\") " pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:26.620917 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:26.620873 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6qk8p" Jul 08 19:46:26.748231 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:26.748162 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6qk8p"] Jul 08 19:46:26.751518 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:46:26.751494 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0479c7a_9963_4c7d_be2b_2860133eb4a4.slice/crio-876f22d0f4a858213427c5d13c4000585b654c5f309462877c94830460e08eb7 WatchSource:0}: Error finding container 876f22d0f4a858213427c5d13c4000585b654c5f309462877c94830460e08eb7: Status 404 returned error can't find the container with id 876f22d0f4a858213427c5d13c4000585b654c5f309462877c94830460e08eb7 Jul 08 19:46:27.577397 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:27.577362 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6qk8p" event={"ID":"a0479c7a-9963-4c7d-be2b-2860133eb4a4","Type":"ContainerStarted","Data":"876f22d0f4a858213427c5d13c4000585b654c5f309462877c94830460e08eb7"} Jul 08 19:46:30.589774 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:30.589738 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6qk8p" event={"ID":"a0479c7a-9963-4c7d-be2b-2860133eb4a4","Type":"ContainerStarted","Data":"0535b0ab316132c765805f3eca53efd349c21d08764e3b8a51194408c55ea322"} Jul 08 19:46:31.127780 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:31.127747 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:31.228397 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:31.228325 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:31.283616 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:31.283577 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:31.431696 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:31.431650 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:31.595670 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:31.595635 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6qk8p" event={"ID":"a0479c7a-9963-4c7d-be2b-2860133eb4a4","Type":"ContainerStarted","Data":"d5a84fc4fa665b7482cad3d6f343aae2eaab5c103057a55f8bfb5a840b9e52b8"} Jul 08 19:46:31.602006 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:31.601974 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:31.658053 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:31.658007 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:31.806663 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:31.806626 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:31.908403 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:31.908320 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:32.202088 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:32.201985 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:32.305076 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:32.305025 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:32.611922 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:32.611887 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:32.789835 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:32.789797 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:33.026787 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.026704 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:33.127839 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.127807 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:33.233722 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.233670 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:33.421575 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.421548 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d9fc557f9-5kxlf"] Jul 08 19:46:33.422921 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.422887 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-fsztn"] Jul 08 19:46:33.426141 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.426118 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-fsztn" Jul 08 19:46:33.429479 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.429458 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 19:46:33.429593 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.429507 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 19:46:33.429593 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.429533 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-pltk9\"" Jul 08 19:46:33.447417 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.447389 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-fsztn"] Jul 08 19:46:33.534339 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.534305 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl7k7\" (UniqueName: \"kubernetes.io/projected/375565ef-c978-4a48-af7f-083f4289557d-kube-api-access-jl7k7\") pod \"downloads-685bfb5f64-fsztn\" (UID: \"375565ef-c978-4a48-af7f-083f4289557d\") " pod="openshift-console/downloads-685bfb5f64-fsztn" Jul 08 19:46:33.568302 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.568269 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:33.604961 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.604926 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6qk8p" event={"ID":"a0479c7a-9963-4c7d-be2b-2860133eb4a4","Type":"ContainerStarted","Data":"843de346e202bacd5a40670988ace23544bc951e11bdad73bf644c0a391b05df"} Jul 08 19:46:33.628928 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.628879 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6qk8p" podStartSLOduration=17.078915325 podStartE2EDuration="23.6288627s" totalImagesPullingTime="6.549947375s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:46:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:46:33.62802138 +0000 UTC m=+275.321330876" watchObservedRunningTime="2026-07-08 19:46:33.6288627 +0000 UTC m=+275.322172185" Jul 08 19:46:33.635281 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.635258 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl7k7\" (UniqueName: \"kubernetes.io/projected/375565ef-c978-4a48-af7f-083f4289557d-kube-api-access-jl7k7\") pod \"downloads-685bfb5f64-fsztn\" (UID: \"375565ef-c978-4a48-af7f-083f4289557d\") " pod="openshift-console/downloads-685bfb5f64-fsztn" Jul 08 19:46:33.646049 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.646015 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl7k7\" (UniqueName: \"kubernetes.io/projected/375565ef-c978-4a48-af7f-083f4289557d-kube-api-access-jl7k7\") pod \"downloads-685bfb5f64-fsztn\" (UID: \"375565ef-c978-4a48-af7f-083f4289557d\") " pod="openshift-console/downloads-685bfb5f64-fsztn" Jul 08 19:46:33.655464 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.655441 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:33.758650 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.758576 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-fsztn" Jul 08 19:46:33.885933 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:33.885903 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-fsztn"] Jul 08 19:46:33.888571 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:46:33.888543 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod375565ef_c978_4a48_af7f_083f4289557d.slice/crio-90357b7ad4dc39db1044bdf2e55b191ae0a819f3caf847dc0d18b9b7e7344e70 WatchSource:0}: Error finding container 90357b7ad4dc39db1044bdf2e55b191ae0a819f3caf847dc0d18b9b7e7344e70: Status 404 returned error can't find the container with id 90357b7ad4dc39db1044bdf2e55b191ae0a819f3caf847dc0d18b9b7e7344e70 Jul 08 19:46:34.041472 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:34.041388 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:34.134234 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:34.134199 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:34.610673 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:34.610634 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-fsztn" event={"ID":"375565ef-c978-4a48-af7f-083f4289557d","Type":"ContainerStarted","Data":"90357b7ad4dc39db1044bdf2e55b191ae0a819f3caf847dc0d18b9b7e7344e70"} Jul 08 19:46:34.912606 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:34.912519 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:34.969348 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:34.969307 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:37.166595 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:37.166561 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-6mr66\" (UID: \"c3fcc756-d506-4c6d-b7fc-39cf568fc9e2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" Jul 08 19:46:37.168959 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:37.168925 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c3fcc756-d506-4c6d-b7fc-39cf568fc9e2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-6mr66\" (UID: \"c3fcc756-d506-4c6d-b7fc-39cf568fc9e2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" Jul 08 19:46:37.467166 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:37.467131 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" Jul 08 19:46:37.602583 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:37.602548 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66"] Jul 08 19:46:37.606279 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:46:37.606252 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3fcc756_d506_4c6d_b7fc_39cf568fc9e2.slice/crio-ac265b49a24f1672b61441301d66b5d37a14423634f6f090caf2c8ee01882187 WatchSource:0}: Error finding container ac265b49a24f1672b61441301d66b5d37a14423634f6f090caf2c8ee01882187: Status 404 returned error can't find the container with id ac265b49a24f1672b61441301d66b5d37a14423634f6f090caf2c8ee01882187 Jul 08 19:46:37.622469 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:37.622427 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" event={"ID":"c3fcc756-d506-4c6d-b7fc-39cf568fc9e2","Type":"ContainerStarted","Data":"ac265b49a24f1672b61441301d66b5d37a14423634f6f090caf2c8ee01882187"} Jul 08 19:46:40.492383 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.492314 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:40.590004 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.589968 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:40.636687 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.636488 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" event={"ID":"c3fcc756-d506-4c6d-b7fc-39cf568fc9e2","Type":"ContainerStarted","Data":"5d8e14def0d45c63d719cb7e10660e6170e09f62461a68238801b7c367a85d39"} Jul 08 19:46:40.656015 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.655950 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6mr66" podStartSLOduration=33.110572126 podStartE2EDuration="35.655930224s" totalImagesPullingTime="2.545358098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:46:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:46:40.654966577 +0000 UTC m=+282.348276064" watchObservedRunningTime="2026-07-08 19:46:40.655930224 +0000 UTC m=+282.349239710" Jul 08 19:46:40.658421 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.658395 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:40.659251 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.659229 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2sgjx"] Jul 08 19:46:40.663016 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.662992 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2sgjx" Jul 08 19:46:40.667288 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.667268 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-4dgtv\"" Jul 08 19:46:40.667438 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.667419 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 08 19:46:40.669793 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.669736 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2sgjx"] Jul 08 19:46:40.797404 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.797312 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/08a324cd-ac8b-41ff-9300-36f99ae759f1-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-2sgjx\" (UID: \"08a324cd-ac8b-41ff-9300-36f99ae759f1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2sgjx" Jul 08 19:46:40.839452 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.839424 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cf69b69db-2jm4m"] Jul 08 19:46:40.842543 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.842527 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:40.845606 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.845573 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 19:46:40.845606 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.845585 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 19:46:40.845606 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.845573 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 19:46:40.845860 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.845658 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-nqqwm\"" Jul 08 19:46:40.845860 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.845665 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 19:46:40.845860 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.845573 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 19:46:40.852436 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.852410 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cf69b69db-2jm4m"] Jul 08 19:46:40.896592 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.896517 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:40.898552 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.898524 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/08a324cd-ac8b-41ff-9300-36f99ae759f1-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-2sgjx\" (UID: \"08a324cd-ac8b-41ff-9300-36f99ae759f1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2sgjx" Jul 08 19:46:40.901338 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.901309 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/08a324cd-ac8b-41ff-9300-36f99ae759f1-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-2sgjx\" (UID: \"08a324cd-ac8b-41ff-9300-36f99ae759f1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2sgjx" Jul 08 19:46:40.975541 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.975500 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2sgjx" Jul 08 19:46:41.000003 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.999905 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d078e834-c512-44ef-99e2-f41b9eb6811f-console-oauth-config\") pod \"console-6cf69b69db-2jm4m\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.000003 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.999953 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d078e834-c512-44ef-99e2-f41b9eb6811f-oauth-serving-cert\") pod \"console-6cf69b69db-2jm4m\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.000003 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:40.999974 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d078e834-c512-44ef-99e2-f41b9eb6811f-console-config\") pod \"console-6cf69b69db-2jm4m\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.000003 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.000000 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqlvx\" (UniqueName: \"kubernetes.io/projected/d078e834-c512-44ef-99e2-f41b9eb6811f-kube-api-access-xqlvx\") pod \"console-6cf69b69db-2jm4m\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.000300 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.000126 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d078e834-c512-44ef-99e2-f41b9eb6811f-console-serving-cert\") pod \"console-6cf69b69db-2jm4m\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.000300 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.000164 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d078e834-c512-44ef-99e2-f41b9eb6811f-service-ca\") pod \"console-6cf69b69db-2jm4m\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.107861 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.101677 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d078e834-c512-44ef-99e2-f41b9eb6811f-console-oauth-config\") pod \"console-6cf69b69db-2jm4m\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.107861 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.101722 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d078e834-c512-44ef-99e2-f41b9eb6811f-oauth-serving-cert\") pod \"console-6cf69b69db-2jm4m\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.107861 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.101751 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d078e834-c512-44ef-99e2-f41b9eb6811f-console-config\") pod \"console-6cf69b69db-2jm4m\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.107861 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.101776 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqlvx\" (UniqueName: \"kubernetes.io/projected/d078e834-c512-44ef-99e2-f41b9eb6811f-kube-api-access-xqlvx\") pod \"console-6cf69b69db-2jm4m\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.107861 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.101823 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d078e834-c512-44ef-99e2-f41b9eb6811f-console-serving-cert\") pod \"console-6cf69b69db-2jm4m\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.107861 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.101848 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d078e834-c512-44ef-99e2-f41b9eb6811f-service-ca\") pod \"console-6cf69b69db-2jm4m\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.107861 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.102664 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d078e834-c512-44ef-99e2-f41b9eb6811f-service-ca\") pod \"console-6cf69b69db-2jm4m\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.107861 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.103616 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d078e834-c512-44ef-99e2-f41b9eb6811f-console-config\") pod \"console-6cf69b69db-2jm4m\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.107861 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.104209 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d078e834-c512-44ef-99e2-f41b9eb6811f-oauth-serving-cert\") pod \"console-6cf69b69db-2jm4m\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.108949 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.108905 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d078e834-c512-44ef-99e2-f41b9eb6811f-console-oauth-config\") pod \"console-6cf69b69db-2jm4m\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.113736 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.113546 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d078e834-c512-44ef-99e2-f41b9eb6811f-console-serving-cert\") pod \"console-6cf69b69db-2jm4m\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.114611 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.114385 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqlvx\" (UniqueName: \"kubernetes.io/projected/d078e834-c512-44ef-99e2-f41b9eb6811f-kube-api-access-xqlvx\") pod \"console-6cf69b69db-2jm4m\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.145423 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.145390 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2sgjx"] Jul 08 19:46:41.147854 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:46:41.147821 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08a324cd_ac8b_41ff_9300_36f99ae759f1.slice/crio-8ee775ebbd13c1d12c2ddfc22a4633751a8ee7be23122f3c076f8777d9013f69 WatchSource:0}: Error finding container 8ee775ebbd13c1d12c2ddfc22a4633751a8ee7be23122f3c076f8777d9013f69: Status 404 returned error can't find the container with id 8ee775ebbd13c1d12c2ddfc22a4633751a8ee7be23122f3c076f8777d9013f69 Jul 08 19:46:41.151733 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.151711 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:46:41.303525 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.303485 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cf69b69db-2jm4m"] Jul 08 19:46:41.307352 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:46:41.307312 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd078e834_c512_44ef_99e2_f41b9eb6811f.slice/crio-6b5c8b40b422e81310f6068df42c5cc5b52a69574813853948772b4aee687574 WatchSource:0}: Error finding container 6b5c8b40b422e81310f6068df42c5cc5b52a69574813853948772b4aee687574: Status 404 returned error can't find the container with id 6b5c8b40b422e81310f6068df42c5cc5b52a69574813853948772b4aee687574 Jul 08 19:46:41.641759 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.641720 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2sgjx" event={"ID":"08a324cd-ac8b-41ff-9300-36f99ae759f1","Type":"ContainerStarted","Data":"8ee775ebbd13c1d12c2ddfc22a4633751a8ee7be23122f3c076f8777d9013f69"} Jul 08 19:46:41.643186 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:41.643152 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cf69b69db-2jm4m" event={"ID":"d078e834-c512-44ef-99e2-f41b9eb6811f","Type":"ContainerStarted","Data":"6b5c8b40b422e81310f6068df42c5cc5b52a69574813853948772b4aee687574"} Jul 08 19:46:43.045267 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:43.045171 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:43.118191 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:43.118147 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:43.429998 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:43.429588 2568 patch_prober.go:28] interesting pod/image-registry-7d9fc557f9-5kxlf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 19:46:43.429998 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:43.429653 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" podUID="9ac68b53-fb73-41f4-9c2f-3a210910c168" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 19:46:43.655527 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:43.655491 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2sgjx" event={"ID":"08a324cd-ac8b-41ff-9300-36f99ae759f1","Type":"ContainerStarted","Data":"103d71c85168d7fdb3b42d95cb407ea6b95dc6ff6e1fa01938bd30f8618f6844"} Jul 08 19:46:43.655868 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:43.655823 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2sgjx" Jul 08 19:46:43.661831 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:43.661784 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2sgjx" Jul 08 19:46:43.673128 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:43.673079 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2sgjx" podStartSLOduration=2.079629039 podStartE2EDuration="3.673064524s" totalImagesPullingTime="1.593435485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:46:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:46:43.671538362 +0000 UTC m=+285.364847862" watchObservedRunningTime="2026-07-08 19:46:43.673064524 +0000 UTC m=+285.366374007" Jul 08 19:46:43.721844 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:43.721757 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:46:43.725263 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:43.725235 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1dead2-f97c-4a89-b5d1-487641ab971a-metrics-tls\") pod \"dns-default-bvr92\" (UID: \"ba1dead2-f97c-4a89-b5d1-487641ab971a\") " pod="openshift-dns/dns-default-bvr92" Jul 08 19:46:43.830205 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:43.830165 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w5s5j\"" Jul 08 19:46:43.837779 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:43.837749 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bvr92" Jul 08 19:46:44.682257 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.682222 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:44.718863 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.718828 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh"] Jul 08 19:46:44.722237 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.722215 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" Jul 08 19:46:44.726758 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.726722 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 19:46:44.726758 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.726745 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 19:46:44.726943 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.726764 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-xqjkg\"" Jul 08 19:46:44.726943 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.726722 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 19:46:44.730426 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.730392 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh"] Jul 08 19:46:44.756932 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.756899 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:44.830809 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.830776 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e3f457a-55de-46ce-8c78-65b0bf1de23d-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-qx8jh\" (UID: \"1e3f457a-55de-46ce-8c78-65b0bf1de23d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" Jul 08 19:46:44.830998 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.830857 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmnnb\" (UniqueName: \"kubernetes.io/projected/1e3f457a-55de-46ce-8c78-65b0bf1de23d-kube-api-access-rmnnb\") pod \"prometheus-operator-567f7b8b7c-qx8jh\" (UID: \"1e3f457a-55de-46ce-8c78-65b0bf1de23d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" Jul 08 19:46:44.830998 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.830897 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1e3f457a-55de-46ce-8c78-65b0bf1de23d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-qx8jh\" (UID: \"1e3f457a-55de-46ce-8c78-65b0bf1de23d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" Jul 08 19:46:44.830998 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.830929 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1e3f457a-55de-46ce-8c78-65b0bf1de23d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-qx8jh\" (UID: \"1e3f457a-55de-46ce-8c78-65b0bf1de23d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" Jul 08 19:46:44.931989 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.931955 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmnnb\" (UniqueName: \"kubernetes.io/projected/1e3f457a-55de-46ce-8c78-65b0bf1de23d-kube-api-access-rmnnb\") pod \"prometheus-operator-567f7b8b7c-qx8jh\" (UID: \"1e3f457a-55de-46ce-8c78-65b0bf1de23d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" Jul 08 19:46:44.932196 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.932006 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1e3f457a-55de-46ce-8c78-65b0bf1de23d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-qx8jh\" (UID: \"1e3f457a-55de-46ce-8c78-65b0bf1de23d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" Jul 08 19:46:44.932196 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.932058 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1e3f457a-55de-46ce-8c78-65b0bf1de23d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-qx8jh\" (UID: \"1e3f457a-55de-46ce-8c78-65b0bf1de23d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" Jul 08 19:46:44.932196 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.932107 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e3f457a-55de-46ce-8c78-65b0bf1de23d-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-qx8jh\" (UID: \"1e3f457a-55de-46ce-8c78-65b0bf1de23d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" Jul 08 19:46:44.932534 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:44.932452 2568 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 08 19:46:44.932663 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:44.932538 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e3f457a-55de-46ce-8c78-65b0bf1de23d-prometheus-operator-tls podName:1e3f457a-55de-46ce-8c78-65b0bf1de23d nodeName:}" failed. No retries permitted until 2026-07-08 19:46:45.432516109 +0000 UTC m=+287.125825575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/1e3f457a-55de-46ce-8c78-65b0bf1de23d-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-qx8jh" (UID: "1e3f457a-55de-46ce-8c78-65b0bf1de23d") : secret "prometheus-operator-tls" not found Jul 08 19:46:44.932866 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.932844 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e3f457a-55de-46ce-8c78-65b0bf1de23d-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-qx8jh\" (UID: \"1e3f457a-55de-46ce-8c78-65b0bf1de23d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" Jul 08 19:46:44.934898 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.934873 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1e3f457a-55de-46ce-8c78-65b0bf1de23d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-qx8jh\" (UID: \"1e3f457a-55de-46ce-8c78-65b0bf1de23d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" Jul 08 19:46:44.944586 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.944552 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmnnb\" (UniqueName: \"kubernetes.io/projected/1e3f457a-55de-46ce-8c78-65b0bf1de23d-kube-api-access-rmnnb\") pod \"prometheus-operator-567f7b8b7c-qx8jh\" (UID: \"1e3f457a-55de-46ce-8c78-65b0bf1de23d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" Jul 08 19:46:44.988866 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:44.988828 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:45.436798 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:45.436763 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1e3f457a-55de-46ce-8c78-65b0bf1de23d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-qx8jh\" (UID: \"1e3f457a-55de-46ce-8c78-65b0bf1de23d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" Jul 08 19:46:45.439224 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:45.439199 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1e3f457a-55de-46ce-8c78-65b0bf1de23d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-qx8jh\" (UID: \"1e3f457a-55de-46ce-8c78-65b0bf1de23d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" Jul 08 19:46:45.453659 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:45.453628 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:45.545596 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:45.545566 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:45.633958 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:45.633919 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" Jul 08 19:46:46.241186 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:46.241149 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:46.302180 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:46.302142 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:48.485474 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:48.485440 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:49.775757 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:49.775717 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:50.446165 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:50.446136 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:50.518587 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:50.518553 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:51.688160 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:51.688118 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cf69b69db-2jm4m" event={"ID":"d078e834-c512-44ef-99e2-f41b9eb6811f","Type":"ContainerStarted","Data":"ca48327787df543f3ea11a82ec406e15ae9a4862a5e0ddd5eaa26ef504e8efb8"} Jul 08 19:46:51.699665 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:51.699643 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bvr92"] Jul 08 19:46:51.701942 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:46:51.701908 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba1dead2_f97c_4a89_b5d1_487641ab971a.slice/crio-aa57886531c7cdfc44ff605a2ae0bc230146e70418f7b257325efb2656dcfe37 WatchSource:0}: Error finding container aa57886531c7cdfc44ff605a2ae0bc230146e70418f7b257325efb2656dcfe37: Status 404 returned error can't find the container with id aa57886531c7cdfc44ff605a2ae0bc230146e70418f7b257325efb2656dcfe37 Jul 08 19:46:51.707685 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:51.707633 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cf69b69db-2jm4m" podStartSLOduration=1.464404913 podStartE2EDuration="11.707613286s" totalImagesPullingTime="10.243208373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:46:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:46:51.707256991 +0000 UTC m=+293.400566477" watchObservedRunningTime="2026-07-08 19:46:51.707613286 +0000 UTC m=+293.400922773" Jul 08 19:46:51.723004 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:51.722966 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh"] Jul 08 19:46:51.725506 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:46:51.725481 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3f457a_55de_46ce_8c78_65b0bf1de23d.slice/crio-3731d73b96655320995f0b54343b87ce4481a9de0e4d720d099ab14d8120e8d7 WatchSource:0}: Error finding container 3731d73b96655320995f0b54343b87ce4481a9de0e4d720d099ab14d8120e8d7: Status 404 returned error can't find the container with id 3731d73b96655320995f0b54343b87ce4481a9de0e4d720d099ab14d8120e8d7 Jul 08 19:46:51.828896 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:51.828855 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-s7hbd" podUID="3b5311a0-d5fa-4994-9071-d861d2a53b09" Jul 08 19:46:52.128063 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:52.127947 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:52.190937 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:52.190895 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:52.696088 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:52.696018 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bvr92" event={"ID":"ba1dead2-f97c-4a89-b5d1-487641ab971a","Type":"ContainerStarted","Data":"aa57886531c7cdfc44ff605a2ae0bc230146e70418f7b257325efb2656dcfe37"} Jul 08 19:46:52.700011 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:52.699908 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-fsztn" event={"ID":"375565ef-c978-4a48-af7f-083f4289557d","Type":"ContainerStarted","Data":"1ea07f25bcfd76c9c3537f0a41e605e0f12e4f0c68c9407223d17918c8852df5"} Jul 08 19:46:52.700011 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:52.699977 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-fsztn" Jul 08 19:46:52.703500 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:52.703437 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" event={"ID":"1e3f457a-55de-46ce-8c78-65b0bf1de23d","Type":"ContainerStarted","Data":"3731d73b96655320995f0b54343b87ce4481a9de0e4d720d099ab14d8120e8d7"} Jul 08 19:46:52.713758 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:52.713730 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-fsztn" Jul 08 19:46:52.721984 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:52.721925 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-fsztn" podStartSLOduration=1.9920969309999998 podStartE2EDuration="19.721903988s" totalImagesPullingTime="17.729807057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:46:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:46:52.718288675 +0000 UTC m=+294.411598162" watchObservedRunningTime="2026-07-08 19:46:52.721903988 +0000 UTC m=+294.415213475" Jul 08 19:46:53.427924 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:53.427892 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:54.644191 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:54.644159 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:54.712589 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:54.712504 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" event={"ID":"1e3f457a-55de-46ce-8c78-65b0bf1de23d","Type":"ContainerStarted","Data":"af2c956757d9bffdda34f186d5bb37eee2478cd34c67ee6b3c249be81b61e337"} Jul 08 19:46:54.744601 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:54.744575 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:54.841721 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:54.841645 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:54.985863 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:54.985809 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:55.087771 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:55.087735 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:55.718987 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:55.718948 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bvr92" event={"ID":"ba1dead2-f97c-4a89-b5d1-487641ab971a","Type":"ContainerStarted","Data":"7b0cf8e50ef5eb65d951f5fd38c7b529ba8082d53e4e4a1d0c281b6c01062e2b"} Jul 08 19:46:55.719452 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:55.719015 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bvr92" event={"ID":"ba1dead2-f97c-4a89-b5d1-487641ab971a","Type":"ContainerStarted","Data":"2479f498a924b3928432a734145bb3fba11f470c6400b4d6bea8fac69306b894"} Jul 08 19:46:55.719452 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:55.719081 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bvr92" Jul 08 19:46:55.729741 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:55.729706 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" event={"ID":"1e3f457a-55de-46ce-8c78-65b0bf1de23d","Type":"ContainerStarted","Data":"a27c911f1e2c2ea955db7ccdcf43d9f76c14e2384e81bf67fba561226adbeaba"} Jul 08 19:46:55.740123 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:55.740075 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bvr92" podStartSLOduration=259.668895208 podStartE2EDuration="4m22.740056212s" totalImagesPullingTime="3.071161004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:46:55.73856573 +0000 UTC m=+297.431875216" watchObservedRunningTime="2026-07-08 19:46:55.740056212 +0000 UTC m=+297.433365690" Jul 08 19:46:55.758785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:55.758721 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qx8jh" podStartSLOduration=9.076409637 podStartE2EDuration="11.758698652s" totalImagesPullingTime="2.682289015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:46:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:46:55.758375146 +0000 UTC m=+297.451684657" watchObservedRunningTime="2026-07-08 19:46:55.758698652 +0000 UTC m=+297.452008157" Jul 08 19:46:55.811685 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:55.811638 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:55.912217 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:55.912178 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:56.178871 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:56.178836 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:56.279587 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:56.279525 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:56.815489 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:56.815455 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cf69b69db-2jm4m"] Jul 08 19:46:57.841204 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:57.841154 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:58.037010 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.036975 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:58.074925 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.074889 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-j44tg"] Jul 08 19:46:58.118394 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.118319 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp"] Jul 08 19:46:58.118567 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.118457 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.121565 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.121541 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 19:46:58.121714 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.121603 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 19:46:58.121714 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.121663 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dzqw7\"" Jul 08 19:46:58.121850 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.121835 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 19:46:58.133393 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.133363 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:58.138800 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.138774 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp"] Jul 08 19:46:58.139078 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.138919 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.140787 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.140758 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-node-exporter-textfile\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.140910 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.140815 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-metrics-client-ca\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.140910 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.140844 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-root\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.140910 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.140869 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pktj\" (UniqueName: \"kubernetes.io/projected/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-kube-api-access-9pktj\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.141113 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.140909 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.141113 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.140963 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-sys\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.141113 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.141086 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-node-exporter-tls\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.141269 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.141136 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-node-exporter-wtmp\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.141269 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.141169 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.141892 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.141872 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 08 19:46:58.142070 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.142028 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 08 19:46:58.142173 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.142155 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-d4hh8\"" Jul 08 19:46:58.142423 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.142394 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 08 19:46:58.206918 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.206879 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:58.241772 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.241733 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.241772 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.241775 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k48rk\" (UniqueName: \"kubernetes.io/projected/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-kube-api-access-k48rk\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.242008 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.241799 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.242008 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.241865 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-node-exporter-tls\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.242008 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.241948 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.242008 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:58.241988 2568 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 19:46:58.242223 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.242014 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.242223 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:58.242092 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-node-exporter-tls podName:ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:58.742065786 +0000 UTC m=+300.435375250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-node-exporter-tls") pod "node-exporter-j44tg" (UID: "ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9") : secret "node-exporter-tls" not found Jul 08 19:46:58.242223 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.242137 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-node-exporter-wtmp\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.242223 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.242165 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.242429 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.242328 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-node-exporter-wtmp\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.242429 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.242373 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-node-exporter-textfile\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.242538 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.242437 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-metrics-client-ca\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.242538 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.242469 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-root\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.242538 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.242494 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pktj\" (UniqueName: \"kubernetes.io/projected/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-kube-api-access-9pktj\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.242538 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.242534 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.242740 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.242635 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-root\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.242740 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.242698 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.242864 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.242740 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-node-exporter-textfile\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.242864 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.242747 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-sys\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.242945 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.242866 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-sys\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.243118 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.243096 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-metrics-client-ca\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.243248 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.243225 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.244966 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.244944 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.253732 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.253704 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pktj\" (UniqueName: \"kubernetes.io/projected/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-kube-api-access-9pktj\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.314254 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.314185 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:58.344221 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.344178 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.344409 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.344255 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.344409 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.344281 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k48rk\" (UniqueName: \"kubernetes.io/projected/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-kube-api-access-k48rk\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.344409 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.344323 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.344580 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.344433 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.344580 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.344474 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.344680 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:58.344586 2568 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 08 19:46:58.344680 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:46:58.344659 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-kube-state-metrics-tls podName:42d0bbf4-6ee3-4b88-9de0-7316f93170f5 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:58.844636998 +0000 UTC m=+300.537946489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-v5nxp" (UID: "42d0bbf4-6ee3-4b88-9de0-7316f93170f5") : secret "kube-state-metrics-tls" not found Jul 08 19:46:58.344825 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.344791 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.345281 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.345255 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.345504 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.345482 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.347180 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.347160 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.354623 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.354600 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k48rk\" (UniqueName: \"kubernetes.io/projected/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-kube-api-access-k48rk\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.414844 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.414760 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:58.447021 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.446977 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" podUID="9ac68b53-fb73-41f4-9c2f-3a210910c168" containerName="registry" containerID="cri-o://f7050887514ecf7020698987aea49c14b0b5ec16bf7b600b4dffb6e3a39e2d25" gracePeriod=30 Jul 08 19:46:58.516052 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.516001 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:58.616237 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.616201 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:58.745290 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.745262 2568 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 19:46:58.747805 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.747766 2568 generic.go:356] "Generic (PLEG): container finished" podID="9ac68b53-fb73-41f4-9c2f-3a210910c168" containerID="f7050887514ecf7020698987aea49c14b0b5ec16bf7b600b4dffb6e3a39e2d25" exitCode=0 Jul 08 19:46:58.747948 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.747831 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" event={"ID":"9ac68b53-fb73-41f4-9c2f-3a210910c168","Type":"ContainerDied","Data":"f7050887514ecf7020698987aea49c14b0b5ec16bf7b600b4dffb6e3a39e2d25"} Jul 08 19:46:58.747948 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.747906 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" event={"ID":"9ac68b53-fb73-41f4-9c2f-3a210910c168","Type":"ContainerDied","Data":"f4d51c2b7129eab329140c8db278072357ea605f786651f80292f57a6cc03ca3"} Jul 08 19:46:58.747948 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.747926 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4d51c2b7129eab329140c8db278072357ea605f786651f80292f57a6cc03ca3" Jul 08 19:46:58.748105 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.748046 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-node-exporter-tls\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.750589 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.750566 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9-node-exporter-tls\") pod \"node-exporter-j44tg\" (UID: \"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9\") " pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:58.754750 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.754724 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:58.848723 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.848563 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-bound-sa-token\") pod \"9ac68b53-fb73-41f4-9c2f-3a210910c168\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " Jul 08 19:46:58.848723 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.848613 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ac68b53-fb73-41f4-9c2f-3a210910c168-trusted-ca\") pod \"9ac68b53-fb73-41f4-9c2f-3a210910c168\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " Jul 08 19:46:58.848723 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.848659 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls\") pod \"9ac68b53-fb73-41f4-9c2f-3a210910c168\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " Jul 08 19:46:58.848723 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.848683 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ac68b53-fb73-41f4-9c2f-3a210910c168-ca-trust-extracted\") pod \"9ac68b53-fb73-41f4-9c2f-3a210910c168\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " Jul 08 19:46:58.848723 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.848716 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-certificates\") pod \"9ac68b53-fb73-41f4-9c2f-3a210910c168\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " Jul 08 19:46:58.849582 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.849119 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsz4v\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-kube-api-access-wsz4v\") pod \"9ac68b53-fb73-41f4-9c2f-3a210910c168\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " Jul 08 19:46:58.849582 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.849155 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9ac68b53-fb73-41f4-9c2f-3a210910c168" (UID: "9ac68b53-fb73-41f4-9c2f-3a210910c168"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:46:58.849582 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.849205 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ac68b53-fb73-41f4-9c2f-3a210910c168-installation-pull-secrets\") pod \"9ac68b53-fb73-41f4-9c2f-3a210910c168\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " Jul 08 19:46:58.849582 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.849263 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ac68b53-fb73-41f4-9c2f-3a210910c168-image-registry-private-configuration\") pod \"9ac68b53-fb73-41f4-9c2f-3a210910c168\" (UID: \"9ac68b53-fb73-41f4-9c2f-3a210910c168\") " Jul 08 19:46:58.849582 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.849256 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ac68b53-fb73-41f4-9c2f-3a210910c168-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9ac68b53-fb73-41f4-9c2f-3a210910c168" (UID: "9ac68b53-fb73-41f4-9c2f-3a210910c168"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:46:58.849582 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.849478 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.849582 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.849551 2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-certificates\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:46:58.849582 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.849569 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ac68b53-fb73-41f4-9c2f-3a210910c168-trusted-ca\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:46:58.851794 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.851747 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9ac68b53-fb73-41f4-9c2f-3a210910c168" (UID: "9ac68b53-fb73-41f4-9c2f-3a210910c168"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:46:58.852760 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.852619 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-kube-api-access-wsz4v" (OuterVolumeSpecName: "kube-api-access-wsz4v") pod "9ac68b53-fb73-41f4-9c2f-3a210910c168" (UID: "9ac68b53-fb73-41f4-9c2f-3a210910c168"). InnerVolumeSpecName "kube-api-access-wsz4v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:46:58.852912 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.852801 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/42d0bbf4-6ee3-4b88-9de0-7316f93170f5-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-v5nxp\" (UID: \"42d0bbf4-6ee3-4b88-9de0-7316f93170f5\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:58.853503 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.853477 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ac68b53-fb73-41f4-9c2f-3a210910c168-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9ac68b53-fb73-41f4-9c2f-3a210910c168" (UID: "9ac68b53-fb73-41f4-9c2f-3a210910c168"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:46:58.853976 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.853946 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9ac68b53-fb73-41f4-9c2f-3a210910c168" (UID: "9ac68b53-fb73-41f4-9c2f-3a210910c168"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:46:58.854251 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.854230 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ac68b53-fb73-41f4-9c2f-3a210910c168-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9ac68b53-fb73-41f4-9c2f-3a210910c168" (UID: "9ac68b53-fb73-41f4-9c2f-3a210910c168"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:46:58.861063 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.861018 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac68b53-fb73-41f4-9c2f-3a210910c168-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9ac68b53-fb73-41f4-9c2f-3a210910c168" (UID: "9ac68b53-fb73-41f4-9c2f-3a210910c168"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:46:58.950772 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.950726 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-registry-tls\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:46:58.950772 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.950774 2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ac68b53-fb73-41f4-9c2f-3a210910c168-ca-trust-extracted\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:46:58.950994 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.950790 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsz4v\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-kube-api-access-wsz4v\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:46:58.950994 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.950800 2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ac68b53-fb73-41f4-9c2f-3a210910c168-installation-pull-secrets\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:46:58.950994 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.950813 2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ac68b53-fb73-41f4-9c2f-3a210910c168-image-registry-private-configuration\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:46:58.950994 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:58.950825 2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ac68b53-fb73-41f4-9c2f-3a210910c168-bound-sa-token\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:46:59.032442 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.032353 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dzqw7\"" Jul 08 19:46:59.039952 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.039916 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-j44tg" Jul 08 19:46:59.047651 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.047615 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:59.053259 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.053228 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-d4hh8\"" Jul 08 19:46:59.061251 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.061226 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" Jul 08 19:46:59.075607 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:46:59.075573 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee8c2b98_1b39_4e81_be6e_c42f1f1b20d9.slice/crio-c9f59bdd7ebb551406aeba8cfbbeff73fc3bd1a715bf646a689a6c610436eb16 WatchSource:0}: Error finding container c9f59bdd7ebb551406aeba8cfbbeff73fc3bd1a715bf646a689a6c610436eb16: Status 404 returned error can't find the container with id c9f59bdd7ebb551406aeba8cfbbeff73fc3bd1a715bf646a689a6c610436eb16 Jul 08 19:46:59.141060 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.140165 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 19:46:59.141060 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.140561 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ac68b53-fb73-41f4-9c2f-3a210910c168" containerName="registry" Jul 08 19:46:59.141060 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.140581 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ac68b53-fb73-41f4-9c2f-3a210910c168" containerName="registry" Jul 08 19:46:59.141060 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.140698 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ac68b53-fb73-41f4-9c2f-3a210910c168" containerName="registry" Jul 08 19:46:59.143081 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.143020 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:59.159807 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.159691 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 19:46:59.159974 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.159852 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.165413 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.165332 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-972sh\"" Jul 08 19:46:59.165561 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.165520 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 19:46:59.165618 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.165579 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 19:46:59.170295 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.169172 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 19:46:59.170295 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.169437 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 19:46:59.170295 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.169669 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 19:46:59.170295 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.169853 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 19:46:59.170295 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.170094 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 19:46:59.170644 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.170307 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 19:46:59.170644 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.170559 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 19:46:59.254351 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.254134 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12241255-34a0-464e-a7a2-c2331a3014b2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.254351 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.254191 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.254351 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.254218 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-web-config\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.254351 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.254237 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.254351 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.254276 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.254351 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.254297 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/12241255-34a0-464e-a7a2-c2331a3014b2-config-out\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.254351 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.254313 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/12241255-34a0-464e-a7a2-c2331a3014b2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.254351 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.254328 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.254351 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.254362 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/12241255-34a0-464e-a7a2-c2331a3014b2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.254924 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.254397 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.254924 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.254432 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12241255-34a0-464e-a7a2-c2331a3014b2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.254924 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.254464 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc2qt\" (UniqueName: \"kubernetes.io/projected/12241255-34a0-464e-a7a2-c2331a3014b2-kube-api-access-lc2qt\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.254924 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.254507 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-config-volume\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.257838 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.257811 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp"] Jul 08 19:46:59.276809 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:46:59.276122 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42d0bbf4_6ee3_4b88_9de0_7316f93170f5.slice/crio-2ce75e3a4322567705597a3804779cb87deffc38df0da652b38cca2349b4e235 WatchSource:0}: Error finding container 2ce75e3a4322567705597a3804779cb87deffc38df0da652b38cca2349b4e235: Status 404 returned error can't find the container with id 2ce75e3a4322567705597a3804779cb87deffc38df0da652b38cca2349b4e235 Jul 08 19:46:59.355103 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.355017 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.355103 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.355098 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/12241255-34a0-464e-a7a2-c2331a3014b2-config-out\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.355365 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.355134 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/12241255-34a0-464e-a7a2-c2331a3014b2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.355365 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.355160 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.355365 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.355222 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:59.355523 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.355232 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/12241255-34a0-464e-a7a2-c2331a3014b2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.355523 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.355489 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.355624 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.355528 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12241255-34a0-464e-a7a2-c2331a3014b2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.355624 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.355557 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc2qt\" (UniqueName: \"kubernetes.io/projected/12241255-34a0-464e-a7a2-c2331a3014b2-kube-api-access-lc2qt\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.355624 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.355601 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-config-volume\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.355776 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.355644 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12241255-34a0-464e-a7a2-c2331a3014b2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.355776 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.355644 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/12241255-34a0-464e-a7a2-c2331a3014b2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.355776 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.355684 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.355776 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.355719 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-web-config\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.355776 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.355748 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.356969 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.356943 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12241255-34a0-464e-a7a2-c2331a3014b2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.358104 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.358076 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12241255-34a0-464e-a7a2-c2331a3014b2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.358472 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.358447 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/12241255-34a0-464e-a7a2-c2331a3014b2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.359231 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.359208 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.359809 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.359760 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.360758 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.360729 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.362156 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.362109 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-web-config\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.362428 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.362356 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.363139 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.362729 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/12241255-34a0-464e-a7a2-c2331a3014b2-config-out\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.363139 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.362770 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.364062 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.364019 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-config-volume\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.365333 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.365313 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc2qt\" (UniqueName: \"kubernetes.io/projected/12241255-34a0-464e-a7a2-c2331a3014b2-kube-api-access-lc2qt\") pod \"alertmanager-main-0\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.482164 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.482124 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:46:59.703372 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.703237 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 19:46:59.705094 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:46:59.705061 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12241255_34a0_464e_a7a2_c2331a3014b2.slice/crio-6b930167108ac633b2bc4c28e3b271cfed5caa7f94bbbfec8144e9bb77e32bae WatchSource:0}: Error finding container 6b930167108ac633b2bc4c28e3b271cfed5caa7f94bbbfec8144e9bb77e32bae: Status 404 returned error can't find the container with id 6b930167108ac633b2bc4c28e3b271cfed5caa7f94bbbfec8144e9bb77e32bae Jul 08 19:46:59.754413 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.754367 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j44tg" event={"ID":"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9","Type":"ContainerStarted","Data":"c9f59bdd7ebb551406aeba8cfbbeff73fc3bd1a715bf646a689a6c610436eb16"} Jul 08 19:46:59.755967 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.755930 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" event={"ID":"42d0bbf4-6ee3-4b88-9de0-7316f93170f5","Type":"ContainerStarted","Data":"2ce75e3a4322567705597a3804779cb87deffc38df0da652b38cca2349b4e235"} Jul 08 19:46:59.757412 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.757391 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d9fc557f9-5kxlf" Jul 08 19:46:59.757602 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.757528 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12241255-34a0-464e-a7a2-c2331a3014b2","Type":"ContainerStarted","Data":"6b930167108ac633b2bc4c28e3b271cfed5caa7f94bbbfec8144e9bb77e32bae"} Jul 08 19:46:59.782892 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.782853 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d9fc557f9-5kxlf"] Jul 08 19:46:59.786502 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:46:59.786478 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7d9fc557f9-5kxlf"] Jul 08 19:47:00.053227 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.053146 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-d7f878767-smnm7"] Jul 08 19:47:00.079978 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.079939 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d7f878767-smnm7"] Jul 08 19:47:00.080171 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.080044 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.083451 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.083398 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 08 19:47:00.083705 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.083683 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 08 19:47:00.085142 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.084136 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 08 19:47:00.085142 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.084399 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 08 19:47:00.085142 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.084603 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-b50kudaq95s8t\"" Jul 08 19:47:00.085142 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.084788 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 08 19:47:00.085142 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.084982 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-cvcs2\"" Jul 08 19:47:00.164102 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.164012 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41a96672-5f5b-4cea-93b9-6a8dbffac896-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.164834 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.164347 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/41a96672-5f5b-4cea-93b9-6a8dbffac896-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.164834 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.164412 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/41a96672-5f5b-4cea-93b9-6a8dbffac896-secret-thanos-querier-tls\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.164834 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.164448 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/41a96672-5f5b-4cea-93b9-6a8dbffac896-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.164834 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.164475 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qwjs\" (UniqueName: \"kubernetes.io/projected/41a96672-5f5b-4cea-93b9-6a8dbffac896-kube-api-access-8qwjs\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.164834 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.164528 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/41a96672-5f5b-4cea-93b9-6a8dbffac896-secret-grpc-tls\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.164834 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.164556 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41a96672-5f5b-4cea-93b9-6a8dbffac896-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.164834 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.164596 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41a96672-5f5b-4cea-93b9-6a8dbffac896-metrics-client-ca\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.266029 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.265978 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41a96672-5f5b-4cea-93b9-6a8dbffac896-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.266227 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.266057 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41a96672-5f5b-4cea-93b9-6a8dbffac896-metrics-client-ca\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.266227 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.266090 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41a96672-5f5b-4cea-93b9-6a8dbffac896-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.266227 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.266171 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/41a96672-5f5b-4cea-93b9-6a8dbffac896-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.266227 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.266207 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/41a96672-5f5b-4cea-93b9-6a8dbffac896-secret-thanos-querier-tls\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.266433 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.266236 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/41a96672-5f5b-4cea-93b9-6a8dbffac896-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.266433 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.266261 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qwjs\" (UniqueName: \"kubernetes.io/projected/41a96672-5f5b-4cea-93b9-6a8dbffac896-kube-api-access-8qwjs\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.266433 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.266319 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/41a96672-5f5b-4cea-93b9-6a8dbffac896-secret-grpc-tls\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.267603 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.267542 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41a96672-5f5b-4cea-93b9-6a8dbffac896-metrics-client-ca\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.270266 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.270086 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/41a96672-5f5b-4cea-93b9-6a8dbffac896-secret-grpc-tls\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.270266 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.270149 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41a96672-5f5b-4cea-93b9-6a8dbffac896-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.270537 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.270348 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/41a96672-5f5b-4cea-93b9-6a8dbffac896-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.270799 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.270767 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/41a96672-5f5b-4cea-93b9-6a8dbffac896-secret-thanos-querier-tls\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.270999 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.270975 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41a96672-5f5b-4cea-93b9-6a8dbffac896-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.272152 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.272126 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/41a96672-5f5b-4cea-93b9-6a8dbffac896-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.275514 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.275479 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qwjs\" (UniqueName: \"kubernetes.io/projected/41a96672-5f5b-4cea-93b9-6a8dbffac896-kube-api-access-8qwjs\") pod \"thanos-querier-d7f878767-smnm7\" (UID: \"41a96672-5f5b-4cea-93b9-6a8dbffac896\") " pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.298281 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.298243 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:00.395767 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.395709 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:00.831432 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:00.831401 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ac68b53-fb73-41f4-9c2f-3a210910c168" path="/var/lib/kubelet/pods/9ac68b53-fb73-41f4-9c2f-3a210910c168/volumes" Jul 08 19:47:01.032758 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:01.032730 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:01.132617 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:01.132539 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:01.152606 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:01.152573 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:47:01.192397 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:01.192369 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:01.349797 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:01.349773 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d7f878767-smnm7"] Jul 08 19:47:01.351350 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:47:01.351325 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41a96672_5f5b_4cea_93b9_6a8dbffac896.slice/crio-c10b73aae505c6feda6af45f3d5a0cfdb3af12a4f40b17c655e265a4b290cd62 WatchSource:0}: Error finding container c10b73aae505c6feda6af45f3d5a0cfdb3af12a4f40b17c655e265a4b290cd62: Status 404 returned error can't find the container with id c10b73aae505c6feda6af45f3d5a0cfdb3af12a4f40b17c655e265a4b290cd62 Jul 08 19:47:01.353948 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:01.353928 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:47:01.501295 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:01.501259 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:01.768485 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:01.768447 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j44tg" event={"ID":"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9","Type":"ContainerStarted","Data":"7575a451fa1e749627357627aa5e181879259f29b094401bc88fe52615ccb2ad"} Jul 08 19:47:01.769850 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:01.769822 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" event={"ID":"41a96672-5f5b-4cea-93b9-6a8dbffac896","Type":"ContainerStarted","Data":"c10b73aae505c6feda6af45f3d5a0cfdb3af12a4f40b17c655e265a4b290cd62"} Jul 08 19:47:01.772205 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:01.772176 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" event={"ID":"42d0bbf4-6ee3-4b88-9de0-7316f93170f5","Type":"ContainerStarted","Data":"53d6dfab176db60cfa190fedde22c0de5602ad54393284f9045bc595bd7e643e"} Jul 08 19:47:02.386474 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.386434 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5445bcf444-xshpk"] Jul 08 19:47:02.401972 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.401903 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5445bcf444-xshpk"] Jul 08 19:47:02.401972 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.402015 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.405101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.405068 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 08 19:47:02.405229 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.405107 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 08 19:47:02.405229 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.405153 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 19:47:02.405308 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.405226 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-d8d9k01ao0jgf\"" Jul 08 19:47:02.405373 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.405357 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-l7pz4\"" Jul 08 19:47:02.405466 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.405441 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 08 19:47:02.486167 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.486128 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ca7c2d56-ffb1-4c71-85cc-62f544725f53-secret-metrics-server-client-certs\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.486378 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.486195 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca7c2d56-ffb1-4c71-85cc-62f544725f53-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.486378 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.486290 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ca7c2d56-ffb1-4c71-85cc-62f544725f53-metrics-server-audit-profiles\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.486378 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.486328 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ca7c2d56-ffb1-4c71-85cc-62f544725f53-audit-log\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.486378 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.486368 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ca7c2d56-ffb1-4c71-85cc-62f544725f53-client-ca-bundle\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.486558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.486398 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nfs2\" (UniqueName: \"kubernetes.io/projected/ca7c2d56-ffb1-4c71-85cc-62f544725f53-kube-api-access-9nfs2\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.486558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.486435 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ca7c2d56-ffb1-4c71-85cc-62f544725f53-secret-metrics-server-tls\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.587110 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.587073 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ca7c2d56-ffb1-4c71-85cc-62f544725f53-metrics-server-audit-profiles\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.587293 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.587117 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ca7c2d56-ffb1-4c71-85cc-62f544725f53-audit-log\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.587293 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.587280 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ca7c2d56-ffb1-4c71-85cc-62f544725f53-client-ca-bundle\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.587365 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.587322 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nfs2\" (UniqueName: \"kubernetes.io/projected/ca7c2d56-ffb1-4c71-85cc-62f544725f53-kube-api-access-9nfs2\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.587365 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.587357 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ca7c2d56-ffb1-4c71-85cc-62f544725f53-secret-metrics-server-tls\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.587445 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.587431 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ca7c2d56-ffb1-4c71-85cc-62f544725f53-secret-metrics-server-client-certs\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.587495 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.587476 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca7c2d56-ffb1-4c71-85cc-62f544725f53-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.587579 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.587560 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ca7c2d56-ffb1-4c71-85cc-62f544725f53-audit-log\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.588195 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.588170 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca7c2d56-ffb1-4c71-85cc-62f544725f53-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.588324 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.588245 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ca7c2d56-ffb1-4c71-85cc-62f544725f53-metrics-server-audit-profiles\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.589957 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.589934 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ca7c2d56-ffb1-4c71-85cc-62f544725f53-secret-metrics-server-tls\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.590424 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.590390 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ca7c2d56-ffb1-4c71-85cc-62f544725f53-client-ca-bundle\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.590470 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.590399 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ca7c2d56-ffb1-4c71-85cc-62f544725f53-secret-metrics-server-client-certs\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.595691 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.595666 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nfs2\" (UniqueName: \"kubernetes.io/projected/ca7c2d56-ffb1-4c71-85cc-62f544725f53-kube-api-access-9nfs2\") pod \"metrics-server-5445bcf444-xshpk\" (UID: \"ca7c2d56-ffb1-4c71-85cc-62f544725f53\") " pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.711393 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.711293 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:02.778883 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.778845 2568 generic.go:356] "Generic (PLEG): container finished" podID="ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9" containerID="7575a451fa1e749627357627aa5e181879259f29b094401bc88fe52615ccb2ad" exitCode=0 Jul 08 19:47:02.779093 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.778926 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j44tg" event={"ID":"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9","Type":"ContainerDied","Data":"7575a451fa1e749627357627aa5e181879259f29b094401bc88fe52615ccb2ad"} Jul 08 19:47:02.779093 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.778991 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j44tg" event={"ID":"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9","Type":"ContainerStarted","Data":"a863ba19e36b4e82155e7048170ac1911e2305a5f597e78433fc6658370002a6"} Jul 08 19:47:02.779093 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.779018 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j44tg" event={"ID":"ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9","Type":"ContainerStarted","Data":"fac97722762d5f51b38f25507f09c0f431de3c478ab7dbeff8edba7c6fd71c73"} Jul 08 19:47:02.780760 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.780709 2568 generic.go:356] "Generic (PLEG): container finished" podID="12241255-34a0-464e-a7a2-c2331a3014b2" containerID="39c26d94c8ca0ada1b0b76865f4ebeef85a50c7798afb415a62fdba07f53ba3f" exitCode=0 Jul 08 19:47:02.780917 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.780780 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12241255-34a0-464e-a7a2-c2331a3014b2","Type":"ContainerDied","Data":"39c26d94c8ca0ada1b0b76865f4ebeef85a50c7798afb415a62fdba07f53ba3f"} Jul 08 19:47:02.785800 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.785772 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" event={"ID":"42d0bbf4-6ee3-4b88-9de0-7316f93170f5","Type":"ContainerStarted","Data":"72c74926e322a561a346c48f065e0348c0fad450578762388b905d4fe2b0c02e"} Jul 08 19:47:02.785911 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.785837 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" event={"ID":"42d0bbf4-6ee3-4b88-9de0-7316f93170f5","Type":"ContainerStarted","Data":"a1f5339362973497070cea2ce5825325ad4bc1da4f22c707f1bedcb33fe87751"} Jul 08 19:47:02.801738 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.801674 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-j44tg" podStartSLOduration=2.57364955 podStartE2EDuration="4.801656492s" totalImagesPullingTime="2.112600244s" totalInitContainerRuntime="115.406698ms" isStatefulPod=true podCreationTimestamp="2026-07-08 19:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:47:02.800561021 +0000 UTC m=+304.493870543" watchObservedRunningTime="2026-07-08 19:47:02.801656492 +0000 UTC m=+304.494966008" Jul 08 19:47:02.824712 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.824624 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v5nxp" podStartSLOduration=2.909123545 podStartE2EDuration="4.824601324s" totalImagesPullingTime="1.915477779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:47:02.824269369 +0000 UTC m=+304.517578855" watchObservedRunningTime="2026-07-08 19:47:02.824601324 +0000 UTC m=+304.517910810" Jul 08 19:47:02.833297 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.833270 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:47:02.848091 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.848053 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:02.867175 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.867137 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-cgkzt"] Jul 08 19:47:02.885086 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:47:02.885051 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca7c2d56_ffb1_4c71_85cc_62f544725f53.slice/crio-7e86d3a5422ca18adc58e6b2395a1c1232cafb08a876df6bd9d54569f0f69074 WatchSource:0}: Error finding container 7e86d3a5422ca18adc58e6b2395a1c1232cafb08a876df6bd9d54569f0f69074: Status 404 returned error can't find the container with id 7e86d3a5422ca18adc58e6b2395a1c1232cafb08a876df6bd9d54569f0f69074 Jul 08 19:47:02.893372 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.893346 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-cgkzt"] Jul 08 19:47:02.893490 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.893389 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5445bcf444-xshpk"] Jul 08 19:47:02.893490 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.893453 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cgkzt" Jul 08 19:47:02.896244 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.896223 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 08 19:47:02.896358 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.896274 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-8pkmf\"" Jul 08 19:47:02.921528 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.921498 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:02.992532 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.992460 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7591c324-9865-49fd-80c6-a87d5a0ff352-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-cgkzt\" (UID: \"7591c324-9865-49fd-80c6-a87d5a0ff352\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cgkzt" Jul 08 19:47:02.992532 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:02.992499 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j49b4\" (UniqueName: \"kubernetes.io/projected/7591c324-9865-49fd-80c6-a87d5a0ff352-kube-api-access-j49b4\") pod \"monitoring-plugin-d5dcff7cf-cgkzt\" (UID: \"7591c324-9865-49fd-80c6-a87d5a0ff352\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cgkzt" Jul 08 19:47:03.093920 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.093885 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7591c324-9865-49fd-80c6-a87d5a0ff352-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-cgkzt\" (UID: \"7591c324-9865-49fd-80c6-a87d5a0ff352\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cgkzt" Jul 08 19:47:03.094096 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.093926 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j49b4\" (UniqueName: \"kubernetes.io/projected/7591c324-9865-49fd-80c6-a87d5a0ff352-kube-api-access-j49b4\") pod \"monitoring-plugin-d5dcff7cf-cgkzt\" (UID: \"7591c324-9865-49fd-80c6-a87d5a0ff352\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cgkzt" Jul 08 19:47:03.096348 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.096322 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7591c324-9865-49fd-80c6-a87d5a0ff352-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-cgkzt\" (UID: \"7591c324-9865-49fd-80c6-a87d5a0ff352\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cgkzt" Jul 08 19:47:03.105280 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.105246 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:03.106596 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.106574 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j49b4\" (UniqueName: \"kubernetes.io/projected/7591c324-9865-49fd-80c6-a87d5a0ff352-kube-api-access-j49b4\") pod \"monitoring-plugin-d5dcff7cf-cgkzt\" (UID: \"7591c324-9865-49fd-80c6-a87d5a0ff352\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cgkzt" Jul 08 19:47:03.169171 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.169141 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:03.204910 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.204878 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cgkzt" Jul 08 19:47:03.273260 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.273183 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-575bcc94c6-fj9zv"] Jul 08 19:47:03.304531 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.302270 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-575bcc94c6-fj9zv"] Jul 08 19:47:03.304531 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.302428 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.308392 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.308361 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 19:47:03.308936 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.308615 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 19:47:03.308936 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.308796 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 19:47:03.309653 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.309627 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-54vp5\"" Jul 08 19:47:03.311513 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.311494 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 19:47:03.311767 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.311750 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 19:47:03.333335 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.333303 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 19:47:03.399737 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.397592 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-cgkzt"] Jul 08 19:47:03.399737 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.398331 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-federate-client-tls\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.399737 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.398376 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-telemeter-client-tls\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.399737 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.398444 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.399737 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.398488 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-serving-certs-ca-bundle\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.399737 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.398538 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.399737 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.398582 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-metrics-client-ca\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.399737 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.398616 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-secret-telemeter-client\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.399737 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.398648 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vrqp\" (UniqueName: \"kubernetes.io/projected/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-kube-api-access-4vrqp\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.403471 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:47:03.403441 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7591c324_9865_49fd_80c6_a87d5a0ff352.slice/crio-48d852c9cb885d92749a5acb78b1df9ec23dfa437d17f8e90143b50747eb25a8 WatchSource:0}: Error finding container 48d852c9cb885d92749a5acb78b1df9ec23dfa437d17f8e90143b50747eb25a8: Status 404 returned error can't find the container with id 48d852c9cb885d92749a5acb78b1df9ec23dfa437d17f8e90143b50747eb25a8 Jul 08 19:47:03.499590 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.499512 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-serving-certs-ca-bundle\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.499775 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.499740 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.499849 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.499810 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-metrics-client-ca\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.499904 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.499848 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-secret-telemeter-client\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.499904 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.499876 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vrqp\" (UniqueName: \"kubernetes.io/projected/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-kube-api-access-4vrqp\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.500011 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.499944 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-federate-client-tls\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.500011 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.499974 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-telemeter-client-tls\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.501227 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.500061 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.501227 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.500337 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-serving-certs-ca-bundle\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.501453 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.501403 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.502128 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.502101 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-metrics-client-ca\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.503957 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.503473 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-telemeter-client-tls\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.503957 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.503474 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.503957 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.503921 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-federate-client-tls\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.504644 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.504609 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-secret-telemeter-client\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.509407 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.509378 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vrqp\" (UniqueName: \"kubernetes.io/projected/b8252151-a32a-4fc4-bb5e-0e9c7c02b07c-kube-api-access-4vrqp\") pod \"telemeter-client-575bcc94c6-fj9zv\" (UID: \"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c\") " pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.622104 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.622016 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" Jul 08 19:47:03.793009 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.792932 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cgkzt" event={"ID":"7591c324-9865-49fd-80c6-a87d5a0ff352","Type":"ContainerStarted","Data":"48d852c9cb885d92749a5acb78b1df9ec23dfa437d17f8e90143b50747eb25a8"} Jul 08 19:47:03.794472 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:03.794383 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" event={"ID":"ca7c2d56-ffb1-4c71-85cc-62f544725f53","Type":"ContainerStarted","Data":"7e86d3a5422ca18adc58e6b2395a1c1232cafb08a876df6bd9d54569f0f69074"} Jul 08 19:47:04.303493 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.303425 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:04.305464 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.303855 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 19:47:04.322289 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.322255 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 19:47:04.322478 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.322441 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.325685 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.325651 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-n9spx\"" Jul 08 19:47:04.326340 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.326109 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 19:47:04.326340 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.326145 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 19:47:04.326340 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.326224 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 19:47:04.326997 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.326978 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 19:47:04.327168 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.327143 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 19:47:04.328068 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.328026 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 19:47:04.328181 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.328163 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 19:47:04.331083 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.330632 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 19:47:04.331083 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.330708 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 19:47:04.331083 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.330736 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-12ed5j18ulim6\"" Jul 08 19:47:04.331083 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.330872 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 19:47:04.331083 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.330911 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 19:47:04.331403 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.331289 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 19:47:04.332398 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.332121 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 19:47:04.335941 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.335919 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 19:47:04.413897 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.413548 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fe44269f-1286-4d41-a2bd-65a10d29a9df-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.413897 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.413601 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-config\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.413897 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.413632 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fe44269f-1286-4d41-a2bd-65a10d29a9df-config-out\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.413897 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.413704 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.413897 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.413760 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.413897 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.413803 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.413897 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.413829 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.413897 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.413856 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.414981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.413902 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss95f\" (UniqueName: \"kubernetes.io/projected/fe44269f-1286-4d41-a2bd-65a10d29a9df-kube-api-access-ss95f\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.414981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.413969 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.414981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.414010 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.414981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.414045 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-web-config\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.414981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.414088 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.414981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.414110 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.414981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.414141 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.414981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.414182 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.414981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.414209 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.414981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.414234 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.414981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.414259 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.414981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.414282 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.515320 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515280 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.515514 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515342 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.515514 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515391 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.515514 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515417 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.515514 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515444 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.515514 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515469 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss95f\" (UniqueName: \"kubernetes.io/projected/fe44269f-1286-4d41-a2bd-65a10d29a9df-kube-api-access-ss95f\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.515514 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515497 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.516710 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515535 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.516710 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515561 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-web-config\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.516710 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515606 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.516710 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515631 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.516710 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515659 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.516710 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515715 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.516710 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515740 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.516710 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515767 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.516710 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515799 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.516710 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515820 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.516710 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515852 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fe44269f-1286-4d41-a2bd-65a10d29a9df-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.516710 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515881 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-config\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.516710 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.515904 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fe44269f-1286-4d41-a2bd-65a10d29a9df-config-out\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.517405 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.516715 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.517753 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.517721 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.518129 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.517955 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.518267 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.518168 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.519341 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.518731 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.519341 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.519007 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.519935 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.519612 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.520207 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.520182 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fe44269f-1286-4d41-a2bd-65a10d29a9df-config-out\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.521832 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.521806 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.523970 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.522847 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.524838 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.524780 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-config\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.527308 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.526905 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.527308 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.526918 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.527308 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.527265 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-web-config\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.528990 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.527862 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fe44269f-1286-4d41-a2bd-65a10d29a9df-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.528990 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.528409 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.528990 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.528740 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.528990 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.528956 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss95f\" (UniqueName: \"kubernetes.io/projected/fe44269f-1286-4d41-a2bd-65a10d29a9df-kube-api-access-ss95f\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.529309 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.529280 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.529972 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.529949 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:04.639533 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:04.639356 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:05.698017 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.697988 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-575bcc94c6-fj9zv"] Jul 08 19:47:05.702721 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:47:05.702694 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8252151_a32a_4fc4_bb5e_0e9c7c02b07c.slice/crio-5f96c3532e865dfb72cb7539e81b6d34f0af43f33157b71d134e8c397cb9aeb9 WatchSource:0}: Error finding container 5f96c3532e865dfb72cb7539e81b6d34f0af43f33157b71d134e8c397cb9aeb9: Status 404 returned error can't find the container with id 5f96c3532e865dfb72cb7539e81b6d34f0af43f33157b71d134e8c397cb9aeb9 Jul 08 19:47:05.719442 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.719412 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 19:47:05.723278 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:47:05.723244 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe44269f_1286_4d41_a2bd_65a10d29a9df.slice/crio-247555fee2fb6f830eba6af7cc1a42ee117dd33b4d9d0a72838d12607c35fc4e WatchSource:0}: Error finding container 247555fee2fb6f830eba6af7cc1a42ee117dd33b4d9d0a72838d12607c35fc4e: Status 404 returned error can't find the container with id 247555fee2fb6f830eba6af7cc1a42ee117dd33b4d9d0a72838d12607c35fc4e Jul 08 19:47:05.737148 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.737125 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bvr92" Jul 08 19:47:05.806870 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.806836 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" event={"ID":"ca7c2d56-ffb1-4c71-85cc-62f544725f53","Type":"ContainerStarted","Data":"00a69f87fb125b018f8e8ae94eb636dc20f94192ca5b30ca578e30d5e13e0e19"} Jul 08 19:47:05.809632 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.809602 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" event={"ID":"41a96672-5f5b-4cea-93b9-6a8dbffac896","Type":"ContainerStarted","Data":"65717d4b8ec6a1eb4872e7f6864db76f74c44bbc54fbc3ed88519dc1a83ebec4"} Jul 08 19:47:05.811512 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.811255 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" event={"ID":"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c","Type":"ContainerStarted","Data":"5f96c3532e865dfb72cb7539e81b6d34f0af43f33157b71d134e8c397cb9aeb9"} Jul 08 19:47:05.813279 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.813254 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe44269f-1286-4d41-a2bd-65a10d29a9df","Type":"ContainerStarted","Data":"247555fee2fb6f830eba6af7cc1a42ee117dd33b4d9d0a72838d12607c35fc4e"} Jul 08 19:47:05.828658 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.828550 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" podStartSLOduration=1.170348421 podStartE2EDuration="3.82852712s" totalImagesPullingTime="2.658178699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:47:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:47:05.828366094 +0000 UTC m=+307.521675582" watchObservedRunningTime="2026-07-08 19:47:05.82852712 +0000 UTC m=+307.521836608" Jul 08 19:47:05.846661 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.846627 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84f7954455-4zmqt"] Jul 08 19:47:05.859627 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.859553 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:05.863602 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.863574 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84f7954455-4zmqt"] Jul 08 19:47:05.870918 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.870893 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 19:47:05.929745 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.929722 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-oauth-serving-cert\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:05.929835 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.929769 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-service-ca\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:05.929889 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.929827 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb19ed1e-bca0-434d-baa7-a79ee82af17e-console-oauth-config\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:05.929989 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.929973 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-trusted-ca-bundle\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:05.930084 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.930021 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-console-config\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:05.930084 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.930062 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w25d2\" (UniqueName: \"kubernetes.io/projected/eb19ed1e-bca0-434d-baa7-a79ee82af17e-kube-api-access-w25d2\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:05.930195 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:05.930144 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb19ed1e-bca0-434d-baa7-a79ee82af17e-console-serving-cert\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:06.031785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.031492 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb19ed1e-bca0-434d-baa7-a79ee82af17e-console-serving-cert\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:06.031785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.031569 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-oauth-serving-cert\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:06.031785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.031607 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-service-ca\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:06.031785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.031642 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb19ed1e-bca0-434d-baa7-a79ee82af17e-console-oauth-config\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:06.031785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.031729 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-trusted-ca-bundle\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:06.031785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.031776 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-console-config\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:06.032338 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.031797 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w25d2\" (UniqueName: \"kubernetes.io/projected/eb19ed1e-bca0-434d-baa7-a79ee82af17e-kube-api-access-w25d2\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:06.032821 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.032740 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-oauth-serving-cert\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:06.033256 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.033226 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-service-ca\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:06.033547 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.033519 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-console-config\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:06.033645 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.033594 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-trusted-ca-bundle\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:06.034601 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.034556 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb19ed1e-bca0-434d-baa7-a79ee82af17e-console-serving-cert\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:06.035543 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.035500 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb19ed1e-bca0-434d-baa7-a79ee82af17e-console-oauth-config\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:06.040685 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.040659 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w25d2\" (UniqueName: \"kubernetes.io/projected/eb19ed1e-bca0-434d-baa7-a79ee82af17e-kube-api-access-w25d2\") pod \"console-84f7954455-4zmqt\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:06.231493 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.231458 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:06.377441 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.377384 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84f7954455-4zmqt"] Jul 08 19:47:06.379491 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:47:06.379468 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb19ed1e_bca0_434d_baa7_a79ee82af17e.slice/crio-ebc17ff9227b8f93754cfc9c0772cc798298077be3d0526b6d53f8a27d7d8f9d WatchSource:0}: Error finding container ebc17ff9227b8f93754cfc9c0772cc798298077be3d0526b6d53f8a27d7d8f9d: Status 404 returned error can't find the container with id ebc17ff9227b8f93754cfc9c0772cc798298077be3d0526b6d53f8a27d7d8f9d Jul 08 19:47:06.639245 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.639166 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:06.824645 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.824600 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cgkzt" event={"ID":"7591c324-9865-49fd-80c6-a87d5a0ff352","Type":"ContainerStarted","Data":"989f5be4b0960cc7cf765982a5333ef180b444e405cd157181ad63d02283e78a"} Jul 08 19:47:06.825200 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.824796 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cgkzt" Jul 08 19:47:06.826508 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.826480 2568 generic.go:356] "Generic (PLEG): container finished" podID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerID="7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841" exitCode=0 Jul 08 19:47:06.833235 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.833207 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe44269f-1286-4d41-a2bd-65a10d29a9df","Type":"ContainerDied","Data":"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841"} Jul 08 19:47:06.833375 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.833308 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cgkzt" Jul 08 19:47:06.833375 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.833335 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84f7954455-4zmqt" event={"ID":"eb19ed1e-bca0-434d-baa7-a79ee82af17e","Type":"ContainerStarted","Data":"8eea115a4819ec48240f50ca9405c56ae3fa43ff87c45cfbc8510d725fb344da"} Jul 08 19:47:06.833375 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.833363 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84f7954455-4zmqt" event={"ID":"eb19ed1e-bca0-434d-baa7-a79ee82af17e","Type":"ContainerStarted","Data":"ebc17ff9227b8f93754cfc9c0772cc798298077be3d0526b6d53f8a27d7d8f9d"} Jul 08 19:47:06.833590 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.833380 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" event={"ID":"41a96672-5f5b-4cea-93b9-6a8dbffac896","Type":"ContainerStarted","Data":"f27c6c9983ed6594fe2239ac5f014898005fef173682f7418968f2fe428fa4eb"} Jul 08 19:47:06.833590 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.833446 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" event={"ID":"41a96672-5f5b-4cea-93b9-6a8dbffac896","Type":"ContainerStarted","Data":"855f74d8661bd55cdda4c0bfea1655d32b16398c2c156b7df0af72a57288b42b"} Jul 08 19:47:06.841951 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.841899 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cgkzt" podStartSLOduration=2.3016694859999998 podStartE2EDuration="4.841880694s" totalImagesPullingTime="2.540211208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:47:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:47:06.840169512 +0000 UTC m=+308.533478999" watchObservedRunningTime="2026-07-08 19:47:06.841880694 +0000 UTC m=+308.535190180" Jul 08 19:47:06.848881 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.848857 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:06.875151 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.875094 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84f7954455-4zmqt" podStartSLOduration=1.875073942 podStartE2EDuration="1.875073942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:47:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:47:06.874689993 +0000 UTC m=+308.567999479" watchObservedRunningTime="2026-07-08 19:47:06.875073942 +0000 UTC m=+308.568383431" Jul 08 19:47:06.907215 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:06.907138 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:07.059831 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:07.059796 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:07.124617 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:07.124586 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:07.842571 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:07.842533 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12241255-34a0-464e-a7a2-c2331a3014b2","Type":"ContainerStarted","Data":"fe79ecbb0d67423ad1883155ab9d3ef3284d0271de0a7d63738628ec0fb41f14"} Jul 08 19:47:07.843086 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:07.842621 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12241255-34a0-464e-a7a2-c2331a3014b2","Type":"ContainerStarted","Data":"31c25318b7595a8274b76d28a5cfb62ef4b76ede6c2d26395c4cfcf6c516b2de"} Jul 08 19:47:07.843086 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:07.842667 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12241255-34a0-464e-a7a2-c2331a3014b2","Type":"ContainerStarted","Data":"1591962a031a0363df57d1e362fedba2d6bb8edcc7188c4f287d78d17d2d4a2f"} Jul 08 19:47:09.856538 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:09.856500 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" event={"ID":"41a96672-5f5b-4cea-93b9-6a8dbffac896","Type":"ContainerStarted","Data":"5884623eb4431e67f91438184b505616acd010e41b2df42b77d9150537f751ca"} Jul 08 19:47:09.857101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:09.856590 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" event={"ID":"41a96672-5f5b-4cea-93b9-6a8dbffac896","Type":"ContainerStarted","Data":"94e041420960e0f5be4787b3d2404db1c15e3a0bca6d76d87cb21c0f3f1d9857"} Jul 08 19:47:09.857101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:09.856632 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" event={"ID":"41a96672-5f5b-4cea-93b9-6a8dbffac896","Type":"ContainerStarted","Data":"5faee64014f6bb458a9862cd77039e7a55c399f5f17d39342ed116d32ac2a4dd"} Jul 08 19:47:09.857101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:09.856717 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:09.859028 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:09.858991 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" event={"ID":"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c","Type":"ContainerStarted","Data":"559a538931c2de8bafb67f725311a66e75a1c59965d8ead03370ed42ccab7a48"} Jul 08 19:47:09.859167 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:09.859075 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" event={"ID":"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c","Type":"ContainerStarted","Data":"e6cbbeef165ca513ff309caa2f0253c26fa799bf65a4e446b5ecc9b4a2746da5"} Jul 08 19:47:09.859167 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:09.859109 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" event={"ID":"b8252151-a32a-4fc4-bb5e-0e9c7c02b07c","Type":"ContainerStarted","Data":"b6cb05bdc7d536c3df2bda647b4666da6711bf6c062e82955fbb0d90e48939ef"} Jul 08 19:47:09.862997 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:09.862963 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12241255-34a0-464e-a7a2-c2331a3014b2","Type":"ContainerStarted","Data":"c2b3c8ea3ae5852847617c6f6c34846759556cc0c113e48108b7b72fd1dd53e1"} Jul 08 19:47:09.863129 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:09.863027 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12241255-34a0-464e-a7a2-c2331a3014b2","Type":"ContainerStarted","Data":"cb638853c215435d33c4d2bc5fcba0cec5d0acf2f8c1d2fb8ab3c0caa454f3f8"} Jul 08 19:47:09.863129 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:09.863079 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12241255-34a0-464e-a7a2-c2331a3014b2","Type":"ContainerStarted","Data":"70ac8d5f2ae82dc4c19fcc179ef09b2eceef9481ae46e3427932194e01c87a22"} Jul 08 19:47:09.881867 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:09.881805 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" podStartSLOduration=2.817052983 podStartE2EDuration="9.881790013s" totalImagesPullingTime="7.06473703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:47:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:47:09.879678269 +0000 UTC m=+311.572987757" watchObservedRunningTime="2026-07-08 19:47:09.881790013 +0000 UTC m=+311.575099498" Jul 08 19:47:09.900888 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:09.900828 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-575bcc94c6-fj9zv" podStartSLOduration=3.668668364 podStartE2EDuration="6.900814595s" totalImagesPullingTime="3.232146231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:47:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:47:09.899546291 +0000 UTC m=+311.592855800" watchObservedRunningTime="2026-07-08 19:47:09.900814595 +0000 UTC m=+311.594124081" Jul 08 19:47:09.928357 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:09.928290 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.018065907 podStartE2EDuration="10.928271199s" totalImagesPullingTime="6.888425802s" totalInitContainerRuntime="21.77949ms" isStatefulPod=false podCreationTimestamp="2026-07-08 19:46:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:47:09.927475159 +0000 UTC m=+311.620784648" watchObservedRunningTime="2026-07-08 19:47:09.928271199 +0000 UTC m=+311.621580684" Jul 08 19:47:10.550398 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.550293 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84f7954455-4zmqt"] Jul 08 19:47:10.579225 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.579185 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8cfc6759c-shwbv"] Jul 08 19:47:10.611796 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.611737 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8cfc6759c-shwbv"] Jul 08 19:47:10.612007 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.611860 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.681951 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.681892 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75725cee-3256-48ee-b4b1-0370ab088770-console-oauth-config\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.682177 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.681983 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-oauth-serving-cert\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.682416 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.682376 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-console-config\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.682635 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.682439 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cxx4\" (UniqueName: \"kubernetes.io/projected/75725cee-3256-48ee-b4b1-0370ab088770-kube-api-access-7cxx4\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.682635 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.682564 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75725cee-3256-48ee-b4b1-0370ab088770-console-serving-cert\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.682635 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.682631 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-trusted-ca-bundle\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.682807 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.682723 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-service-ca\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.784227 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.784186 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75725cee-3256-48ee-b4b1-0370ab088770-console-oauth-config\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.784227 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.784229 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-oauth-serving-cert\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.784476 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.784260 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-console-config\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.784476 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.784289 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cxx4\" (UniqueName: \"kubernetes.io/projected/75725cee-3256-48ee-b4b1-0370ab088770-kube-api-access-7cxx4\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.784476 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.784332 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75725cee-3256-48ee-b4b1-0370ab088770-console-serving-cert\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.784476 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.784379 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-trusted-ca-bundle\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.784476 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.784458 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-service-ca\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.785205 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.785127 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-oauth-serving-cert\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.785327 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.785275 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-console-config\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.785464 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.785437 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-service-ca\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.785884 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.785864 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-trusted-ca-bundle\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.787485 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.787454 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75725cee-3256-48ee-b4b1-0370ab088770-console-serving-cert\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.787602 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.787539 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75725cee-3256-48ee-b4b1-0370ab088770-console-oauth-config\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.795475 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.795423 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cxx4\" (UniqueName: \"kubernetes.io/projected/75725cee-3256-48ee-b4b1-0370ab088770-kube-api-access-7cxx4\") pod \"console-8cfc6759c-shwbv\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.876613 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.876571 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-d7f878767-smnm7" Jul 08 19:47:10.928009 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.927972 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:10.996873 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:10.995329 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:11.096347 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:11.096177 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:11.119303 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:11.119269 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8cfc6759c-shwbv"] Jul 08 19:47:11.122893 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:47:11.122860 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75725cee_3256_48ee_b4b1_0370ab088770.slice/crio-8b1c319396f1a5f349e94d9ec58ee05d278721a39df733be0b75b75fc400d9d6 WatchSource:0}: Error finding container 8b1c319396f1a5f349e94d9ec58ee05d278721a39df733be0b75b75fc400d9d6: Status 404 returned error can't find the container with id 8b1c319396f1a5f349e94d9ec58ee05d278721a39df733be0b75b75fc400d9d6 Jul 08 19:47:11.567374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:11.567345 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:11.677961 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:11.677866 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:11.877473 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:11.877438 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe44269f-1286-4d41-a2bd-65a10d29a9df","Type":"ContainerStarted","Data":"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd"} Jul 08 19:47:11.877887 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:11.877523 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe44269f-1286-4d41-a2bd-65a10d29a9df","Type":"ContainerStarted","Data":"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee"} Jul 08 19:47:11.877887 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:11.877570 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe44269f-1286-4d41-a2bd-65a10d29a9df","Type":"ContainerStarted","Data":"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d"} Jul 08 19:47:11.879097 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:11.879073 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8cfc6759c-shwbv" event={"ID":"75725cee-3256-48ee-b4b1-0370ab088770","Type":"ContainerStarted","Data":"dd46eb83ed14a8b08e866920930104f7f9bf5e81d98b65edf7df5d24040401cc"} Jul 08 19:47:11.879214 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:11.879114 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8cfc6759c-shwbv" event={"ID":"75725cee-3256-48ee-b4b1-0370ab088770","Type":"ContainerStarted","Data":"8b1c319396f1a5f349e94d9ec58ee05d278721a39df733be0b75b75fc400d9d6"} Jul 08 19:47:11.903396 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:11.903318 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8cfc6759c-shwbv" podStartSLOduration=1.9032980849999999 podStartE2EDuration="1.903298085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:47:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:47:11.901779928 +0000 UTC m=+313.595089416" watchObservedRunningTime="2026-07-08 19:47:11.903298085 +0000 UTC m=+313.596607574" Jul 08 19:47:12.887127 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:12.887089 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe44269f-1286-4d41-a2bd-65a10d29a9df","Type":"ContainerStarted","Data":"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c"} Jul 08 19:47:12.887658 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:12.887163 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe44269f-1286-4d41-a2bd-65a10d29a9df","Type":"ContainerStarted","Data":"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7"} Jul 08 19:47:12.887658 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:12.887191 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe44269f-1286-4d41-a2bd-65a10d29a9df","Type":"ContainerStarted","Data":"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744"} Jul 08 19:47:12.923377 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:12.921906 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.34780737 podStartE2EDuration="8.921882838s" totalImagesPullingTime="4.557241098s" totalInitContainerRuntime="16.83437ms" isStatefulPod=false podCreationTimestamp="2026-07-08 19:47:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:47:12.917429743 +0000 UTC m=+314.610739249" watchObservedRunningTime="2026-07-08 19:47:12.921882838 +0000 UTC m=+314.615192325" Jul 08 19:47:14.640167 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:14.640136 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:47:14.855827 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:14.855796 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:15.921724 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:15.921691 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:15.988255 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:15.988226 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:16.232345 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:16.232267 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:18.100596 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:18.100564 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:18.164546 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:18.164514 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:20.929123 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:20.929091 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:20.929685 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:20.929160 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:20.933506 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:20.933484 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:21.840106 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:21.840060 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6cf69b69db-2jm4m" podUID="d078e834-c512-44ef-99e2-f41b9eb6811f" containerName="console" containerID="cri-o://ca48327787df543f3ea11a82ec406e15ae9a4862a5e0ddd5eaa26ef504e8efb8" gracePeriod=15 Jul 08 19:47:21.933067 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:21.933017 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:47:22.109556 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.109528 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:22.131079 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.131028 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:47:22.207112 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.207078 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:22.301001 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.300965 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d078e834-c512-44ef-99e2-f41b9eb6811f-service-ca\") pod \"d078e834-c512-44ef-99e2-f41b9eb6811f\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " Jul 08 19:47:22.301210 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.301022 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d078e834-c512-44ef-99e2-f41b9eb6811f-console-serving-cert\") pod \"d078e834-c512-44ef-99e2-f41b9eb6811f\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " Jul 08 19:47:22.301210 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.301090 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d078e834-c512-44ef-99e2-f41b9eb6811f-console-config\") pod \"d078e834-c512-44ef-99e2-f41b9eb6811f\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " Jul 08 19:47:22.301210 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.301119 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d078e834-c512-44ef-99e2-f41b9eb6811f-console-oauth-config\") pod \"d078e834-c512-44ef-99e2-f41b9eb6811f\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " Jul 08 19:47:22.301210 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.301149 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqlvx\" (UniqueName: \"kubernetes.io/projected/d078e834-c512-44ef-99e2-f41b9eb6811f-kube-api-access-xqlvx\") pod \"d078e834-c512-44ef-99e2-f41b9eb6811f\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " Jul 08 19:47:22.301210 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.301201 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d078e834-c512-44ef-99e2-f41b9eb6811f-oauth-serving-cert\") pod \"d078e834-c512-44ef-99e2-f41b9eb6811f\" (UID: \"d078e834-c512-44ef-99e2-f41b9eb6811f\") " Jul 08 19:47:22.301463 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.301344 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d078e834-c512-44ef-99e2-f41b9eb6811f-service-ca" (OuterVolumeSpecName: "service-ca") pod "d078e834-c512-44ef-99e2-f41b9eb6811f" (UID: "d078e834-c512-44ef-99e2-f41b9eb6811f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:47:22.301516 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.301486 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d078e834-c512-44ef-99e2-f41b9eb6811f-service-ca\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:47:22.301574 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.301552 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d078e834-c512-44ef-99e2-f41b9eb6811f-console-config" (OuterVolumeSpecName: "console-config") pod "d078e834-c512-44ef-99e2-f41b9eb6811f" (UID: "d078e834-c512-44ef-99e2-f41b9eb6811f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:47:22.301574 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.301559 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d078e834-c512-44ef-99e2-f41b9eb6811f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d078e834-c512-44ef-99e2-f41b9eb6811f" (UID: "d078e834-c512-44ef-99e2-f41b9eb6811f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:47:22.303480 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.303449 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d078e834-c512-44ef-99e2-f41b9eb6811f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d078e834-c512-44ef-99e2-f41b9eb6811f" (UID: "d078e834-c512-44ef-99e2-f41b9eb6811f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:47:22.303590 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.303523 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d078e834-c512-44ef-99e2-f41b9eb6811f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d078e834-c512-44ef-99e2-f41b9eb6811f" (UID: "d078e834-c512-44ef-99e2-f41b9eb6811f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:47:22.303590 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.303533 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d078e834-c512-44ef-99e2-f41b9eb6811f-kube-api-access-xqlvx" (OuterVolumeSpecName: "kube-api-access-xqlvx") pod "d078e834-c512-44ef-99e2-f41b9eb6811f" (UID: "d078e834-c512-44ef-99e2-f41b9eb6811f"). InnerVolumeSpecName "kube-api-access-xqlvx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:47:22.307335 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.307313 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:22.402999 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.402908 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqlvx\" (UniqueName: \"kubernetes.io/projected/d078e834-c512-44ef-99e2-f41b9eb6811f-kube-api-access-xqlvx\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:47:22.402999 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.402937 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d078e834-c512-44ef-99e2-f41b9eb6811f-oauth-serving-cert\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:47:22.402999 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.402952 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d078e834-c512-44ef-99e2-f41b9eb6811f-console-serving-cert\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:47:22.402999 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.402963 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d078e834-c512-44ef-99e2-f41b9eb6811f-console-config\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:47:22.402999 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.402975 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d078e834-c512-44ef-99e2-f41b9eb6811f-console-oauth-config\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:47:22.712091 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.711999 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:22.712224 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.712098 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:22.934213 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.934180 2568 generic.go:356] "Generic (PLEG): container finished" podID="d078e834-c512-44ef-99e2-f41b9eb6811f" containerID="ca48327787df543f3ea11a82ec406e15ae9a4862a5e0ddd5eaa26ef504e8efb8" exitCode=0 Jul 08 19:47:22.934622 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.934245 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cf69b69db-2jm4m" Jul 08 19:47:22.934622 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.934269 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cf69b69db-2jm4m" event={"ID":"d078e834-c512-44ef-99e2-f41b9eb6811f","Type":"ContainerDied","Data":"ca48327787df543f3ea11a82ec406e15ae9a4862a5e0ddd5eaa26ef504e8efb8"} Jul 08 19:47:22.934622 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.934327 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cf69b69db-2jm4m" event={"ID":"d078e834-c512-44ef-99e2-f41b9eb6811f","Type":"ContainerDied","Data":"6b5c8b40b422e81310f6068df42c5cc5b52a69574813853948772b4aee687574"} Jul 08 19:47:22.934622 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.934364 2568 scope.go:117] "RemoveContainer" containerID="ca48327787df543f3ea11a82ec406e15ae9a4862a5e0ddd5eaa26ef504e8efb8" Jul 08 19:47:22.952387 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.952364 2568 scope.go:117] "RemoveContainer" containerID="ca48327787df543f3ea11a82ec406e15ae9a4862a5e0ddd5eaa26ef504e8efb8" Jul 08 19:47:22.952705 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:47:22.952681 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca48327787df543f3ea11a82ec406e15ae9a4862a5e0ddd5eaa26ef504e8efb8\": container with ID starting with ca48327787df543f3ea11a82ec406e15ae9a4862a5e0ddd5eaa26ef504e8efb8 not found: ID does not exist" containerID="ca48327787df543f3ea11a82ec406e15ae9a4862a5e0ddd5eaa26ef504e8efb8" Jul 08 19:47:22.952784 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.952714 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca48327787df543f3ea11a82ec406e15ae9a4862a5e0ddd5eaa26ef504e8efb8"} err="failed to get container status \"ca48327787df543f3ea11a82ec406e15ae9a4862a5e0ddd5eaa26ef504e8efb8\": rpc error: code = NotFound desc = could not find container \"ca48327787df543f3ea11a82ec406e15ae9a4862a5e0ddd5eaa26ef504e8efb8\": container with ID starting with ca48327787df543f3ea11a82ec406e15ae9a4862a5e0ddd5eaa26ef504e8efb8 not found: ID does not exist" Jul 08 19:47:22.952837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.952822 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cf69b69db-2jm4m"] Jul 08 19:47:22.955782 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:22.955762 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6cf69b69db-2jm4m"] Jul 08 19:47:24.831706 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:24.831659 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d078e834-c512-44ef-99e2-f41b9eb6811f" path="/var/lib/kubelet/pods/d078e834-c512-44ef-99e2-f41b9eb6811f/volumes" Jul 08 19:47:33.113974 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:33.113918 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:33.212470 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:33.212434 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:47:35.579840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:35.579770 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-84f7954455-4zmqt" podUID="eb19ed1e-bca0-434d-baa7-a79ee82af17e" containerName="console" containerID="cri-o://8eea115a4819ec48240f50ca9405c56ae3fa43ff87c45cfbc8510d725fb344da" gracePeriod=15 Jul 08 19:47:35.915289 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:35.915256 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:35.999350 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:35.999318 2568 generic.go:356] "Generic (PLEG): container finished" podID="eb19ed1e-bca0-434d-baa7-a79ee82af17e" containerID="8eea115a4819ec48240f50ca9405c56ae3fa43ff87c45cfbc8510d725fb344da" exitCode=0 Jul 08 19:47:35.999551 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:35.999364 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84f7954455-4zmqt" Jul 08 19:47:35.999551 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:35.999420 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84f7954455-4zmqt" event={"ID":"eb19ed1e-bca0-434d-baa7-a79ee82af17e","Type":"ContainerDied","Data":"8eea115a4819ec48240f50ca9405c56ae3fa43ff87c45cfbc8510d725fb344da"} Jul 08 19:47:35.999551 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:35.999484 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84f7954455-4zmqt" event={"ID":"eb19ed1e-bca0-434d-baa7-a79ee82af17e","Type":"ContainerDied","Data":"ebc17ff9227b8f93754cfc9c0772cc798298077be3d0526b6d53f8a27d7d8f9d"} Jul 08 19:47:35.999551 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:35.999509 2568 scope.go:117] "RemoveContainer" containerID="8eea115a4819ec48240f50ca9405c56ae3fa43ff87c45cfbc8510d725fb344da" Jul 08 19:47:36.008841 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.008819 2568 scope.go:117] "RemoveContainer" containerID="8eea115a4819ec48240f50ca9405c56ae3fa43ff87c45cfbc8510d725fb344da" Jul 08 19:47:36.009188 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:47:36.009168 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eea115a4819ec48240f50ca9405c56ae3fa43ff87c45cfbc8510d725fb344da\": container with ID starting with 8eea115a4819ec48240f50ca9405c56ae3fa43ff87c45cfbc8510d725fb344da not found: ID does not exist" containerID="8eea115a4819ec48240f50ca9405c56ae3fa43ff87c45cfbc8510d725fb344da" Jul 08 19:47:36.009256 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.009198 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eea115a4819ec48240f50ca9405c56ae3fa43ff87c45cfbc8510d725fb344da"} err="failed to get container status \"8eea115a4819ec48240f50ca9405c56ae3fa43ff87c45cfbc8510d725fb344da\": rpc error: code = NotFound desc = could not find container \"8eea115a4819ec48240f50ca9405c56ae3fa43ff87c45cfbc8510d725fb344da\": container with ID starting with 8eea115a4819ec48240f50ca9405c56ae3fa43ff87c45cfbc8510d725fb344da not found: ID does not exist" Jul 08 19:47:36.037275 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.037242 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w25d2\" (UniqueName: \"kubernetes.io/projected/eb19ed1e-bca0-434d-baa7-a79ee82af17e-kube-api-access-w25d2\") pod \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " Jul 08 19:47:36.037441 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.037285 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-oauth-serving-cert\") pod \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " Jul 08 19:47:36.037441 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.037310 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb19ed1e-bca0-434d-baa7-a79ee82af17e-console-oauth-config\") pod \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " Jul 08 19:47:36.037441 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.037414 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-console-config\") pod \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " Jul 08 19:47:36.037600 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.037458 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-trusted-ca-bundle\") pod \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " Jul 08 19:47:36.037600 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.037532 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-service-ca\") pod \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " Jul 08 19:47:36.037600 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.037556 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb19ed1e-bca0-434d-baa7-a79ee82af17e-console-serving-cert\") pod \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\" (UID: \"eb19ed1e-bca0-434d-baa7-a79ee82af17e\") " Jul 08 19:47:36.037750 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.037663 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "eb19ed1e-bca0-434d-baa7-a79ee82af17e" (UID: "eb19ed1e-bca0-434d-baa7-a79ee82af17e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:47:36.037750 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.037732 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-console-config" (OuterVolumeSpecName: "console-config") pod "eb19ed1e-bca0-434d-baa7-a79ee82af17e" (UID: "eb19ed1e-bca0-434d-baa7-a79ee82af17e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:47:36.038019 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.037931 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "eb19ed1e-bca0-434d-baa7-a79ee82af17e" (UID: "eb19ed1e-bca0-434d-baa7-a79ee82af17e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:47:36.038019 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.037924 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-service-ca" (OuterVolumeSpecName: "service-ca") pod "eb19ed1e-bca0-434d-baa7-a79ee82af17e" (UID: "eb19ed1e-bca0-434d-baa7-a79ee82af17e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:47:36.038019 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.037941 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-oauth-serving-cert\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:47:36.038019 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.037992 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-console-config\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:47:36.039670 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.039634 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb19ed1e-bca0-434d-baa7-a79ee82af17e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "eb19ed1e-bca0-434d-baa7-a79ee82af17e" (UID: "eb19ed1e-bca0-434d-baa7-a79ee82af17e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:47:36.039766 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.039662 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb19ed1e-bca0-434d-baa7-a79ee82af17e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "eb19ed1e-bca0-434d-baa7-a79ee82af17e" (UID: "eb19ed1e-bca0-434d-baa7-a79ee82af17e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:47:36.039766 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.039694 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb19ed1e-bca0-434d-baa7-a79ee82af17e-kube-api-access-w25d2" (OuterVolumeSpecName: "kube-api-access-w25d2") pod "eb19ed1e-bca0-434d-baa7-a79ee82af17e" (UID: "eb19ed1e-bca0-434d-baa7-a79ee82af17e"). InnerVolumeSpecName "kube-api-access-w25d2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:47:36.139713 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.139609 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w25d2\" (UniqueName: \"kubernetes.io/projected/eb19ed1e-bca0-434d-baa7-a79ee82af17e-kube-api-access-w25d2\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:47:36.139713 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.139654 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb19ed1e-bca0-434d-baa7-a79ee82af17e-console-oauth-config\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:47:36.139713 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.139671 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-trusted-ca-bundle\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:47:36.139713 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.139687 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb19ed1e-bca0-434d-baa7-a79ee82af17e-service-ca\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:47:36.139713 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.139702 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb19ed1e-bca0-434d-baa7-a79ee82af17e-console-serving-cert\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:47:36.327821 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.327783 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84f7954455-4zmqt"] Jul 08 19:47:36.330659 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.330626 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84f7954455-4zmqt"] Jul 08 19:47:36.836944 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:36.836886 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb19ed1e-bca0-434d-baa7-a79ee82af17e" path="/var/lib/kubelet/pods/eb19ed1e-bca0-434d-baa7-a79ee82af17e/volumes" Jul 08 19:47:42.716646 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:42.716614 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:47:42.720500 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:47:42.720480 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5445bcf444-xshpk" Jul 08 19:48:01.243780 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:01.243684 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:01.310537 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:01.310505 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:04.640555 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:04.640519 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:04.659922 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:04.659885 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:05.147783 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:05.147757 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:12.683723 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:12.683681 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs\") pod \"network-metrics-daemon-s7hbd\" (UID: \"3b5311a0-d5fa-4994-9071-d861d2a53b09\") " pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:48:12.686001 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:12.685980 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b5311a0-d5fa-4994-9071-d861d2a53b09-metrics-certs\") pod \"network-metrics-daemon-s7hbd\" (UID: \"3b5311a0-d5fa-4994-9071-d861d2a53b09\") " pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:48:12.737733 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:12.737702 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dpn4j\"" Jul 08 19:48:12.745178 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:12.745155 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7hbd" Jul 08 19:48:12.877129 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:12.877103 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s7hbd"] Jul 08 19:48:12.879307 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:48:12.879280 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b5311a0_d5fa_4994_9071_d861d2a53b09.slice/crio-7b8793a6cd9fefccce63aca756159c17c2bd5144cdc0c5abee81b9e5e2353779 WatchSource:0}: Error finding container 7b8793a6cd9fefccce63aca756159c17c2bd5144cdc0c5abee81b9e5e2353779: Status 404 returned error can't find the container with id 7b8793a6cd9fefccce63aca756159c17c2bd5144cdc0c5abee81b9e5e2353779 Jul 08 19:48:13.166419 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:13.166385 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s7hbd" event={"ID":"3b5311a0-d5fa-4994-9071-d861d2a53b09","Type":"ContainerStarted","Data":"7b8793a6cd9fefccce63aca756159c17c2bd5144cdc0c5abee81b9e5e2353779"} Jul 08 19:48:14.173167 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:14.173067 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s7hbd" event={"ID":"3b5311a0-d5fa-4994-9071-d861d2a53b09","Type":"ContainerStarted","Data":"ab19bfe3456b4ff284e52344b0b2cb3277f69204a049563259d19231de5ba1df"} Jul 08 19:48:14.173167 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:14.173133 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s7hbd" event={"ID":"3b5311a0-d5fa-4994-9071-d861d2a53b09","Type":"ContainerStarted","Data":"b8ee8cb2f39b0b5b73f0b435d72fab5289d8776bbbd7e67c04b305cbf712ba14"} Jul 08 19:48:14.191374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:14.191314 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-s7hbd" podStartSLOduration=374.241048286 podStartE2EDuration="6m15.191292891s" totalImagesPullingTime="950.244605ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:48:14.190957547 +0000 UTC m=+375.884267058" watchObservedRunningTime="2026-07-08 19:48:14.191292891 +0000 UTC m=+375.884602380" Jul 08 19:48:14.195261 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:14.195236 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:14.257995 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:14.257941 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:14.312561 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:14.312537 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:14.366778 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:14.366751 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:18.450280 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:18.450243 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 19:48:18.450726 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:18.450559 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="alertmanager" containerID="cri-o://1591962a031a0363df57d1e362fedba2d6bb8edcc7188c4f287d78d17d2d4a2f" gracePeriod=120 Jul 08 19:48:18.450726 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:18.450650 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="kube-rbac-proxy-web" containerID="cri-o://fe79ecbb0d67423ad1883155ab9d3ef3284d0271de0a7d63738628ec0fb41f14" gracePeriod=120 Jul 08 19:48:18.450726 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:18.450674 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="prom-label-proxy" containerID="cri-o://c2b3c8ea3ae5852847617c6f6c34846759556cc0c113e48108b7b72fd1dd53e1" gracePeriod=120 Jul 08 19:48:18.450919 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:18.450712 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="config-reloader" containerID="cri-o://31c25318b7595a8274b76d28a5cfb62ef4b76ede6c2d26395c4cfcf6c516b2de" gracePeriod=120 Jul 08 19:48:18.450919 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:18.450724 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="kube-rbac-proxy" containerID="cri-o://70ac8d5f2ae82dc4c19fcc179ef09b2eceef9481ae46e3427932194e01c87a22" gracePeriod=120 Jul 08 19:48:18.450919 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:18.450637 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="kube-rbac-proxy-metric" containerID="cri-o://cb638853c215435d33c4d2bc5fcba0cec5d0acf2f8c1d2fb8ab3c0caa454f3f8" gracePeriod=120 Jul 08 19:48:19.204770 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.204738 2568 generic.go:356] "Generic (PLEG): container finished" podID="12241255-34a0-464e-a7a2-c2331a3014b2" containerID="c2b3c8ea3ae5852847617c6f6c34846759556cc0c113e48108b7b72fd1dd53e1" exitCode=0 Jul 08 19:48:19.204770 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.204762 2568 generic.go:356] "Generic (PLEG): container finished" podID="12241255-34a0-464e-a7a2-c2331a3014b2" containerID="cb638853c215435d33c4d2bc5fcba0cec5d0acf2f8c1d2fb8ab3c0caa454f3f8" exitCode=0 Jul 08 19:48:19.204770 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.204769 2568 generic.go:356] "Generic (PLEG): container finished" podID="12241255-34a0-464e-a7a2-c2331a3014b2" containerID="70ac8d5f2ae82dc4c19fcc179ef09b2eceef9481ae46e3427932194e01c87a22" exitCode=0 Jul 08 19:48:19.204770 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.204775 2568 generic.go:356] "Generic (PLEG): container finished" podID="12241255-34a0-464e-a7a2-c2331a3014b2" containerID="31c25318b7595a8274b76d28a5cfb62ef4b76ede6c2d26395c4cfcf6c516b2de" exitCode=0 Jul 08 19:48:19.204770 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.204781 2568 generic.go:356] "Generic (PLEG): container finished" podID="12241255-34a0-464e-a7a2-c2331a3014b2" containerID="1591962a031a0363df57d1e362fedba2d6bb8edcc7188c4f287d78d17d2d4a2f" exitCode=0 Jul 08 19:48:19.205080 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.204806 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12241255-34a0-464e-a7a2-c2331a3014b2","Type":"ContainerDied","Data":"c2b3c8ea3ae5852847617c6f6c34846759556cc0c113e48108b7b72fd1dd53e1"} Jul 08 19:48:19.205080 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.204878 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12241255-34a0-464e-a7a2-c2331a3014b2","Type":"ContainerDied","Data":"cb638853c215435d33c4d2bc5fcba0cec5d0acf2f8c1d2fb8ab3c0caa454f3f8"} Jul 08 19:48:19.205080 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.204908 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12241255-34a0-464e-a7a2-c2331a3014b2","Type":"ContainerDied","Data":"70ac8d5f2ae82dc4c19fcc179ef09b2eceef9481ae46e3427932194e01c87a22"} Jul 08 19:48:19.205080 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.204931 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12241255-34a0-464e-a7a2-c2331a3014b2","Type":"ContainerDied","Data":"31c25318b7595a8274b76d28a5cfb62ef4b76ede6c2d26395c4cfcf6c516b2de"} Jul 08 19:48:19.205080 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.204956 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12241255-34a0-464e-a7a2-c2331a3014b2","Type":"ContainerDied","Data":"1591962a031a0363df57d1e362fedba2d6bb8edcc7188c4f287d78d17d2d4a2f"} Jul 08 19:48:19.719688 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.719666 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:19.851607 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.851575 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-config-volume\") pod \"12241255-34a0-464e-a7a2-c2331a3014b2\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " Jul 08 19:48:19.851802 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.851619 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"12241255-34a0-464e-a7a2-c2331a3014b2\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " Jul 08 19:48:19.851802 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.851644 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12241255-34a0-464e-a7a2-c2331a3014b2-alertmanager-trusted-ca-bundle\") pod \"12241255-34a0-464e-a7a2-c2331a3014b2\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " Jul 08 19:48:19.851802 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.851674 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-kube-rbac-proxy-web\") pod \"12241255-34a0-464e-a7a2-c2331a3014b2\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " Jul 08 19:48:19.851802 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.851706 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/12241255-34a0-464e-a7a2-c2331a3014b2-config-out\") pod \"12241255-34a0-464e-a7a2-c2331a3014b2\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " Jul 08 19:48:19.851802 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.851729 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12241255-34a0-464e-a7a2-c2331a3014b2-metrics-client-ca\") pod \"12241255-34a0-464e-a7a2-c2331a3014b2\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " Jul 08 19:48:19.851802 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.851752 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lc2qt\" (UniqueName: \"kubernetes.io/projected/12241255-34a0-464e-a7a2-c2331a3014b2-kube-api-access-lc2qt\") pod \"12241255-34a0-464e-a7a2-c2331a3014b2\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " Jul 08 19:48:19.851802 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.851791 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-main-tls\") pod \"12241255-34a0-464e-a7a2-c2331a3014b2\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " Jul 08 19:48:19.852205 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.851842 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-web-config\") pod \"12241255-34a0-464e-a7a2-c2331a3014b2\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " Jul 08 19:48:19.852205 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.851869 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-cluster-tls-config\") pod \"12241255-34a0-464e-a7a2-c2331a3014b2\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " Jul 08 19:48:19.852205 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.851893 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/12241255-34a0-464e-a7a2-c2331a3014b2-tls-assets\") pod \"12241255-34a0-464e-a7a2-c2331a3014b2\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " Jul 08 19:48:19.852205 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.851924 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-kube-rbac-proxy\") pod \"12241255-34a0-464e-a7a2-c2331a3014b2\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " Jul 08 19:48:19.852205 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.851979 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/12241255-34a0-464e-a7a2-c2331a3014b2-alertmanager-main-db\") pod \"12241255-34a0-464e-a7a2-c2331a3014b2\" (UID: \"12241255-34a0-464e-a7a2-c2331a3014b2\") " Jul 08 19:48:19.852205 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.852134 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12241255-34a0-464e-a7a2-c2331a3014b2-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "12241255-34a0-464e-a7a2-c2331a3014b2" (UID: "12241255-34a0-464e-a7a2-c2331a3014b2"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:48:19.852496 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.852365 2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12241255-34a0-464e-a7a2-c2331a3014b2-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:19.852496 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.852388 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12241255-34a0-464e-a7a2-c2331a3014b2-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "12241255-34a0-464e-a7a2-c2331a3014b2" (UID: "12241255-34a0-464e-a7a2-c2331a3014b2"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:48:19.852496 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.852399 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12241255-34a0-464e-a7a2-c2331a3014b2-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "12241255-34a0-464e-a7a2-c2331a3014b2" (UID: "12241255-34a0-464e-a7a2-c2331a3014b2"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:48:19.854470 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.854412 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "12241255-34a0-464e-a7a2-c2331a3014b2" (UID: "12241255-34a0-464e-a7a2-c2331a3014b2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:48:19.855014 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.854977 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12241255-34a0-464e-a7a2-c2331a3014b2-config-out" (OuterVolumeSpecName: "config-out") pod "12241255-34a0-464e-a7a2-c2331a3014b2" (UID: "12241255-34a0-464e-a7a2-c2331a3014b2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:48:19.855288 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.855267 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12241255-34a0-464e-a7a2-c2331a3014b2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "12241255-34a0-464e-a7a2-c2331a3014b2" (UID: "12241255-34a0-464e-a7a2-c2331a3014b2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:48:19.855384 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.855327 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-config-volume" (OuterVolumeSpecName: "config-volume") pod "12241255-34a0-464e-a7a2-c2331a3014b2" (UID: "12241255-34a0-464e-a7a2-c2331a3014b2"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:48:19.855552 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.855533 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "12241255-34a0-464e-a7a2-c2331a3014b2" (UID: "12241255-34a0-464e-a7a2-c2331a3014b2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:48:19.856332 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.856302 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "12241255-34a0-464e-a7a2-c2331a3014b2" (UID: "12241255-34a0-464e-a7a2-c2331a3014b2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:48:19.856569 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.856551 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12241255-34a0-464e-a7a2-c2331a3014b2-kube-api-access-lc2qt" (OuterVolumeSpecName: "kube-api-access-lc2qt") pod "12241255-34a0-464e-a7a2-c2331a3014b2" (UID: "12241255-34a0-464e-a7a2-c2331a3014b2"). InnerVolumeSpecName "kube-api-access-lc2qt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:48:19.856660 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.856563 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "12241255-34a0-464e-a7a2-c2331a3014b2" (UID: "12241255-34a0-464e-a7a2-c2331a3014b2"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:48:19.859485 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.859455 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "12241255-34a0-464e-a7a2-c2331a3014b2" (UID: "12241255-34a0-464e-a7a2-c2331a3014b2"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:48:19.866660 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.866634 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-web-config" (OuterVolumeSpecName: "web-config") pod "12241255-34a0-464e-a7a2-c2331a3014b2" (UID: "12241255-34a0-464e-a7a2-c2331a3014b2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:48:19.953709 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.953675 2568 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-config-volume\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:19.953709 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.953705 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:19.953709 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.953717 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:19.953923 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.953727 2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/12241255-34a0-464e-a7a2-c2331a3014b2-config-out\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:19.953923 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.953736 2568 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12241255-34a0-464e-a7a2-c2331a3014b2-metrics-client-ca\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:19.953923 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.953745 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lc2qt\" (UniqueName: \"kubernetes.io/projected/12241255-34a0-464e-a7a2-c2331a3014b2-kube-api-access-lc2qt\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:19.953923 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.953755 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-main-tls\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:19.953923 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.953764 2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-web-config\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:19.953923 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.953772 2568 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-cluster-tls-config\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:19.953923 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.953780 2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/12241255-34a0-464e-a7a2-c2331a3014b2-tls-assets\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:19.953923 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.953789 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12241255-34a0-464e-a7a2-c2331a3014b2-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:19.953923 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:19.953797 2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/12241255-34a0-464e-a7a2-c2331a3014b2-alertmanager-main-db\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:20.212927 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.212838 2568 generic.go:356] "Generic (PLEG): container finished" podID="12241255-34a0-464e-a7a2-c2331a3014b2" containerID="fe79ecbb0d67423ad1883155ab9d3ef3284d0271de0a7d63738628ec0fb41f14" exitCode=0 Jul 08 19:48:20.213120 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.212925 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.213120 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.212922 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12241255-34a0-464e-a7a2-c2331a3014b2","Type":"ContainerDied","Data":"fe79ecbb0d67423ad1883155ab9d3ef3284d0271de0a7d63738628ec0fb41f14"} Jul 08 19:48:20.213120 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.213083 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12241255-34a0-464e-a7a2-c2331a3014b2","Type":"ContainerDied","Data":"6b930167108ac633b2bc4c28e3b271cfed5caa7f94bbbfec8144e9bb77e32bae"} Jul 08 19:48:20.213120 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.213117 2568 scope.go:117] "RemoveContainer" containerID="c2b3c8ea3ae5852847617c6f6c34846759556cc0c113e48108b7b72fd1dd53e1" Jul 08 19:48:20.221277 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.221251 2568 scope.go:117] "RemoveContainer" containerID="cb638853c215435d33c4d2bc5fcba0cec5d0acf2f8c1d2fb8ab3c0caa454f3f8" Jul 08 19:48:20.228525 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.228506 2568 scope.go:117] "RemoveContainer" containerID="70ac8d5f2ae82dc4c19fcc179ef09b2eceef9481ae46e3427932194e01c87a22" Jul 08 19:48:20.235341 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.235324 2568 scope.go:117] "RemoveContainer" containerID="fe79ecbb0d67423ad1883155ab9d3ef3284d0271de0a7d63738628ec0fb41f14" Jul 08 19:48:20.238070 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.238024 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 19:48:20.242948 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.242922 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 19:48:20.244379 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.244360 2568 scope.go:117] "RemoveContainer" containerID="31c25318b7595a8274b76d28a5cfb62ef4b76ede6c2d26395c4cfcf6c516b2de" Jul 08 19:48:20.251307 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.251292 2568 scope.go:117] "RemoveContainer" containerID="1591962a031a0363df57d1e362fedba2d6bb8edcc7188c4f287d78d17d2d4a2f" Jul 08 19:48:20.261768 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.261748 2568 scope.go:117] "RemoveContainer" containerID="39c26d94c8ca0ada1b0b76865f4ebeef85a50c7798afb415a62fdba07f53ba3f" Jul 08 19:48:20.267323 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267298 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 19:48:20.267620 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267608 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="kube-rbac-proxy" Jul 08 19:48:20.267664 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267622 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="kube-rbac-proxy" Jul 08 19:48:20.267664 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267631 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d078e834-c512-44ef-99e2-f41b9eb6811f" containerName="console" Jul 08 19:48:20.267664 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267638 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d078e834-c512-44ef-99e2-f41b9eb6811f" containerName="console" Jul 08 19:48:20.267664 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267648 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="prom-label-proxy" Jul 08 19:48:20.267664 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267653 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="prom-label-proxy" Jul 08 19:48:20.267664 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267664 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb19ed1e-bca0-434d-baa7-a79ee82af17e" containerName="console" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267669 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb19ed1e-bca0-434d-baa7-a79ee82af17e" containerName="console" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267675 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="kube-rbac-proxy-metric" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267680 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="kube-rbac-proxy-metric" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267687 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="alertmanager" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267692 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="alertmanager" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267700 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="config-reloader" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267704 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="config-reloader" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267712 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="kube-rbac-proxy-web" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267716 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="kube-rbac-proxy-web" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267729 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="init-config-reloader" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267734 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="init-config-reloader" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267782 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb19ed1e-bca0-434d-baa7-a79ee82af17e" containerName="console" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267790 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="kube-rbac-proxy-web" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267797 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="prom-label-proxy" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267803 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d078e834-c512-44ef-99e2-f41b9eb6811f" containerName="console" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267809 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="config-reloader" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267815 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="kube-rbac-proxy-metric" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267822 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="alertmanager" Jul 08 19:48:20.267840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.267826 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" containerName="kube-rbac-proxy" Jul 08 19:48:20.271049 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.271017 2568 scope.go:117] "RemoveContainer" containerID="c2b3c8ea3ae5852847617c6f6c34846759556cc0c113e48108b7b72fd1dd53e1" Jul 08 19:48:20.271366 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:48:20.271346 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2b3c8ea3ae5852847617c6f6c34846759556cc0c113e48108b7b72fd1dd53e1\": container with ID starting with c2b3c8ea3ae5852847617c6f6c34846759556cc0c113e48108b7b72fd1dd53e1 not found: ID does not exist" containerID="c2b3c8ea3ae5852847617c6f6c34846759556cc0c113e48108b7b72fd1dd53e1" Jul 08 19:48:20.271431 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.271372 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2b3c8ea3ae5852847617c6f6c34846759556cc0c113e48108b7b72fd1dd53e1"} err="failed to get container status \"c2b3c8ea3ae5852847617c6f6c34846759556cc0c113e48108b7b72fd1dd53e1\": rpc error: code = NotFound desc = could not find container \"c2b3c8ea3ae5852847617c6f6c34846759556cc0c113e48108b7b72fd1dd53e1\": container with ID starting with c2b3c8ea3ae5852847617c6f6c34846759556cc0c113e48108b7b72fd1dd53e1 not found: ID does not exist" Jul 08 19:48:20.271431 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.271393 2568 scope.go:117] "RemoveContainer" containerID="cb638853c215435d33c4d2bc5fcba0cec5d0acf2f8c1d2fb8ab3c0caa454f3f8" Jul 08 19:48:20.271559 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.271544 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.271625 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:48:20.271609 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb638853c215435d33c4d2bc5fcba0cec5d0acf2f8c1d2fb8ab3c0caa454f3f8\": container with ID starting with cb638853c215435d33c4d2bc5fcba0cec5d0acf2f8c1d2fb8ab3c0caa454f3f8 not found: ID does not exist" containerID="cb638853c215435d33c4d2bc5fcba0cec5d0acf2f8c1d2fb8ab3c0caa454f3f8" Jul 08 19:48:20.271666 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.271630 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb638853c215435d33c4d2bc5fcba0cec5d0acf2f8c1d2fb8ab3c0caa454f3f8"} err="failed to get container status \"cb638853c215435d33c4d2bc5fcba0cec5d0acf2f8c1d2fb8ab3c0caa454f3f8\": rpc error: code = NotFound desc = could not find container \"cb638853c215435d33c4d2bc5fcba0cec5d0acf2f8c1d2fb8ab3c0caa454f3f8\": container with ID starting with cb638853c215435d33c4d2bc5fcba0cec5d0acf2f8c1d2fb8ab3c0caa454f3f8 not found: ID does not exist" Jul 08 19:48:20.271666 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.271646 2568 scope.go:117] "RemoveContainer" containerID="70ac8d5f2ae82dc4c19fcc179ef09b2eceef9481ae46e3427932194e01c87a22" Jul 08 19:48:20.271931 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:48:20.271910 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70ac8d5f2ae82dc4c19fcc179ef09b2eceef9481ae46e3427932194e01c87a22\": container with ID starting with 70ac8d5f2ae82dc4c19fcc179ef09b2eceef9481ae46e3427932194e01c87a22 not found: ID does not exist" containerID="70ac8d5f2ae82dc4c19fcc179ef09b2eceef9481ae46e3427932194e01c87a22" Jul 08 19:48:20.272012 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.271939 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70ac8d5f2ae82dc4c19fcc179ef09b2eceef9481ae46e3427932194e01c87a22"} err="failed to get container status \"70ac8d5f2ae82dc4c19fcc179ef09b2eceef9481ae46e3427932194e01c87a22\": rpc error: code = NotFound desc = could not find container \"70ac8d5f2ae82dc4c19fcc179ef09b2eceef9481ae46e3427932194e01c87a22\": container with ID starting with 70ac8d5f2ae82dc4c19fcc179ef09b2eceef9481ae46e3427932194e01c87a22 not found: ID does not exist" Jul 08 19:48:20.272012 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.271961 2568 scope.go:117] "RemoveContainer" containerID="fe79ecbb0d67423ad1883155ab9d3ef3284d0271de0a7d63738628ec0fb41f14" Jul 08 19:48:20.272223 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:48:20.272204 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe79ecbb0d67423ad1883155ab9d3ef3284d0271de0a7d63738628ec0fb41f14\": container with ID starting with fe79ecbb0d67423ad1883155ab9d3ef3284d0271de0a7d63738628ec0fb41f14 not found: ID does not exist" containerID="fe79ecbb0d67423ad1883155ab9d3ef3284d0271de0a7d63738628ec0fb41f14" Jul 08 19:48:20.272291 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.272230 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe79ecbb0d67423ad1883155ab9d3ef3284d0271de0a7d63738628ec0fb41f14"} err="failed to get container status \"fe79ecbb0d67423ad1883155ab9d3ef3284d0271de0a7d63738628ec0fb41f14\": rpc error: code = NotFound desc = could not find container \"fe79ecbb0d67423ad1883155ab9d3ef3284d0271de0a7d63738628ec0fb41f14\": container with ID starting with fe79ecbb0d67423ad1883155ab9d3ef3284d0271de0a7d63738628ec0fb41f14 not found: ID does not exist" Jul 08 19:48:20.272291 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.272251 2568 scope.go:117] "RemoveContainer" containerID="31c25318b7595a8274b76d28a5cfb62ef4b76ede6c2d26395c4cfcf6c516b2de" Jul 08 19:48:20.272517 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:48:20.272491 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31c25318b7595a8274b76d28a5cfb62ef4b76ede6c2d26395c4cfcf6c516b2de\": container with ID starting with 31c25318b7595a8274b76d28a5cfb62ef4b76ede6c2d26395c4cfcf6c516b2de not found: ID does not exist" containerID="31c25318b7595a8274b76d28a5cfb62ef4b76ede6c2d26395c4cfcf6c516b2de" Jul 08 19:48:20.272617 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.272523 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31c25318b7595a8274b76d28a5cfb62ef4b76ede6c2d26395c4cfcf6c516b2de"} err="failed to get container status \"31c25318b7595a8274b76d28a5cfb62ef4b76ede6c2d26395c4cfcf6c516b2de\": rpc error: code = NotFound desc = could not find container \"31c25318b7595a8274b76d28a5cfb62ef4b76ede6c2d26395c4cfcf6c516b2de\": container with ID starting with 31c25318b7595a8274b76d28a5cfb62ef4b76ede6c2d26395c4cfcf6c516b2de not found: ID does not exist" Jul 08 19:48:20.272617 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.272544 2568 scope.go:117] "RemoveContainer" containerID="1591962a031a0363df57d1e362fedba2d6bb8edcc7188c4f287d78d17d2d4a2f" Jul 08 19:48:20.272829 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:48:20.272812 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1591962a031a0363df57d1e362fedba2d6bb8edcc7188c4f287d78d17d2d4a2f\": container with ID starting with 1591962a031a0363df57d1e362fedba2d6bb8edcc7188c4f287d78d17d2d4a2f not found: ID does not exist" containerID="1591962a031a0363df57d1e362fedba2d6bb8edcc7188c4f287d78d17d2d4a2f" Jul 08 19:48:20.272880 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.272836 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1591962a031a0363df57d1e362fedba2d6bb8edcc7188c4f287d78d17d2d4a2f"} err="failed to get container status \"1591962a031a0363df57d1e362fedba2d6bb8edcc7188c4f287d78d17d2d4a2f\": rpc error: code = NotFound desc = could not find container \"1591962a031a0363df57d1e362fedba2d6bb8edcc7188c4f287d78d17d2d4a2f\": container with ID starting with 1591962a031a0363df57d1e362fedba2d6bb8edcc7188c4f287d78d17d2d4a2f not found: ID does not exist" Jul 08 19:48:20.272880 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.272856 2568 scope.go:117] "RemoveContainer" containerID="39c26d94c8ca0ada1b0b76865f4ebeef85a50c7798afb415a62fdba07f53ba3f" Jul 08 19:48:20.273144 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:48:20.273127 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39c26d94c8ca0ada1b0b76865f4ebeef85a50c7798afb415a62fdba07f53ba3f\": container with ID starting with 39c26d94c8ca0ada1b0b76865f4ebeef85a50c7798afb415a62fdba07f53ba3f not found: ID does not exist" containerID="39c26d94c8ca0ada1b0b76865f4ebeef85a50c7798afb415a62fdba07f53ba3f" Jul 08 19:48:20.273196 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.273150 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39c26d94c8ca0ada1b0b76865f4ebeef85a50c7798afb415a62fdba07f53ba3f"} err="failed to get container status \"39c26d94c8ca0ada1b0b76865f4ebeef85a50c7798afb415a62fdba07f53ba3f\": rpc error: code = NotFound desc = could not find container \"39c26d94c8ca0ada1b0b76865f4ebeef85a50c7798afb415a62fdba07f53ba3f\": container with ID starting with 39c26d94c8ca0ada1b0b76865f4ebeef85a50c7798afb415a62fdba07f53ba3f not found: ID does not exist" Jul 08 19:48:20.275953 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.275935 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 19:48:20.276064 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.275994 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 19:48:20.276121 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.276072 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-972sh\"" Jul 08 19:48:20.276121 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.276080 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 19:48:20.276121 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.276097 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 19:48:20.276358 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.276331 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 19:48:20.276768 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.276748 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 19:48:20.276870 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.276749 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 19:48:20.276870 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.276756 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 19:48:20.280093 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.280073 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 19:48:20.283638 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.283620 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 19:48:20.359956 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.359927 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-web-config\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.360172 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.359975 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-config-volume\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.360172 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.360002 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.360172 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.360024 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.360172 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.360069 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-config-out\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.360172 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.360085 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.360172 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.360132 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.360397 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.360199 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.360397 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.360220 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.360397 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.360241 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.360397 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.360274 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.360397 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.360310 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.360397 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.360328 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrmrk\" (UniqueName: \"kubernetes.io/projected/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-kube-api-access-rrmrk\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.460752 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.460715 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.460752 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.460753 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrmrk\" (UniqueName: \"kubernetes.io/projected/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-kube-api-access-rrmrk\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.461003 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.460781 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-web-config\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.461003 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.460830 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-config-volume\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.461003 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.460874 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.461003 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.460899 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.461003 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.460931 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-config-out\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.461003 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.460954 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.461003 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.460983 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.461366 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.461020 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.461366 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.461069 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.461366 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.461095 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.461366 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.461144 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.463161 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.462070 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.463161 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.462697 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.463687 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.463575 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.463839 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.463818 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-config-out\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.463909 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.463877 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.464024 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.463999 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-web-config\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.464163 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.464143 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.464374 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.464349 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.464475 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.464385 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.465822 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.465799 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.465905 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.465863 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-config-volume\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.466182 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.466165 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.470663 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.470644 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrmrk\" (UniqueName: \"kubernetes.io/projected/a64384f1-f9ac-4caf-96f7-0fd1fc2722e6-kube-api-access-rrmrk\") pod \"alertmanager-main-0\" (UID: \"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.583493 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.583453 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:48:20.721179 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.721153 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 19:48:20.723484 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:48:20.723450 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda64384f1_f9ac_4caf_96f7_0fd1fc2722e6.slice/crio-2286399bf8845b32bbc92e60ad4c691dfa435a4ec46ec9ed86f64e7acc87dae0 WatchSource:0}: Error finding container 2286399bf8845b32bbc92e60ad4c691dfa435a4ec46ec9ed86f64e7acc87dae0: Status 404 returned error can't find the container with id 2286399bf8845b32bbc92e60ad4c691dfa435a4ec46ec9ed86f64e7acc87dae0 Jul 08 19:48:20.831158 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:20.831121 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12241255-34a0-464e-a7a2-c2331a3014b2" path="/var/lib/kubelet/pods/12241255-34a0-464e-a7a2-c2331a3014b2/volumes" Jul 08 19:48:21.218565 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:21.218532 2568 generic.go:356] "Generic (PLEG): container finished" podID="a64384f1-f9ac-4caf-96f7-0fd1fc2722e6" containerID="d42b55d6e6de7ed867b872983f450182066c18e4905cb61faa40236dc2606d07" exitCode=0 Jul 08 19:48:21.218741 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:21.218614 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6","Type":"ContainerDied","Data":"d42b55d6e6de7ed867b872983f450182066c18e4905cb61faa40236dc2606d07"} Jul 08 19:48:21.218741 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:21.218691 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6","Type":"ContainerStarted","Data":"2286399bf8845b32bbc92e60ad4c691dfa435a4ec46ec9ed86f64e7acc87dae0"} Jul 08 19:48:22.228169 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:22.228137 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6","Type":"ContainerStarted","Data":"7e1dbe4bd7bb0217110bc6aa88d07016c4d403350d48d9cf2d8527274297110b"} Jul 08 19:48:22.228525 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:22.228204 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6","Type":"ContainerStarted","Data":"c4ade1161019e8e50db677d995a43adc2341657cc508ea4c39741fd827a4fc8c"} Jul 08 19:48:22.228525 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:22.228231 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6","Type":"ContainerStarted","Data":"d13dfb974b110f4c2b3ca988b832bf4fb43158207dbd23fb1a90323fd7f86959"} Jul 08 19:48:22.228525 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:22.228256 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6","Type":"ContainerStarted","Data":"3913e722e8005ead2a71ab0531ddefc81bc55f0b772dc96b6ec4c8dc947d6e4d"} Jul 08 19:48:22.228525 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:22.228278 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6","Type":"ContainerStarted","Data":"ac60b222c02b14a6b0a70e69ff001d9bf23c25ab9145d95b3ed9d7e11c154538"} Jul 08 19:48:22.228525 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:22.228301 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a64384f1-f9ac-4caf-96f7-0fd1fc2722e6","Type":"ContainerStarted","Data":"2b81cccdce068933f9687214273dcb85fd0422b9275499bc2097187794db69d3"} Jul 08 19:48:22.255952 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:22.255899 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.241212028 podStartE2EDuration="2.255883191s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.671163ms" isStatefulPod=false podCreationTimestamp="2026-07-08 19:48:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:48:22.255509738 +0000 UTC m=+383.948819262" watchObservedRunningTime="2026-07-08 19:48:22.255883191 +0000 UTC m=+383.949192676" Jul 08 19:48:22.670369 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:22.670335 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 19:48:22.671151 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:22.670699 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="prometheus" containerID="cri-o://8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d" gracePeriod=600 Jul 08 19:48:22.671151 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:22.670861 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="kube-rbac-proxy" containerID="cri-o://4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7" gracePeriod=600 Jul 08 19:48:22.671151 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:22.670886 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="thanos-sidecar" containerID="cri-o://382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd" gracePeriod=600 Jul 08 19:48:22.671151 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:22.670865 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="kube-rbac-proxy-web" containerID="cri-o://d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744" gracePeriod=600 Jul 08 19:48:22.671151 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:22.670962 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="config-reloader" containerID="cri-o://8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee" gracePeriod=600 Jul 08 19:48:22.671151 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:22.671058 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="kube-rbac-proxy-thanos" containerID="cri-o://73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c" gracePeriod=600 Jul 08 19:48:22.940789 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:22.940766 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.093712 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.093666 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-grpc-tls\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.093712 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.093718 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-trusted-ca-bundle\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.093923 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.093744 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-thanos-prometheus-http-client-file\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.093923 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.093772 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-kube-rbac-proxy\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.093923 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.093831 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-db\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.093923 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.093882 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-rulefiles-2\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.093923 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.093914 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fe44269f-1286-4d41-a2bd-65a10d29a9df-tls-assets\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.094202 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.093943 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ss95f\" (UniqueName: \"kubernetes.io/projected/fe44269f-1286-4d41-a2bd-65a10d29a9df-kube-api-access-ss95f\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.094202 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.093968 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-rulefiles-0\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.094202 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.094000 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-metrics-client-certs\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.094202 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.094025 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.094202 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.094103 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.094202 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.094132 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-web-config\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.094202 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.094197 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-configmap-metrics-client-ca\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.094537 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.094231 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-config\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.094537 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.094258 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-configmap-kubelet-serving-ca-bundle\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.094537 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.094282 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-rulefiles-1\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.094537 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.094325 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-configmap-serving-certs-ca-bundle\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.094537 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.094369 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fe44269f-1286-4d41-a2bd-65a10d29a9df-config-out\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.094537 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.094395 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-prometheus-k8s-tls\") pod \"fe44269f-1286-4d41-a2bd-65a10d29a9df\" (UID: \"fe44269f-1286-4d41-a2bd-65a10d29a9df\") " Jul 08 19:48:23.094537 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.094504 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:48:23.094849 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.094724 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.095026 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.095001 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:48:23.095124 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.095081 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:48:23.096471 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.096439 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:48:23.096583 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.096487 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe44269f-1286-4d41-a2bd-65a10d29a9df-kube-api-access-ss95f" (OuterVolumeSpecName: "kube-api-access-ss95f") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "kube-api-access-ss95f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:48:23.096583 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.096546 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:48:23.096583 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.096563 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe44269f-1286-4d41-a2bd-65a10d29a9df-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:48:23.096789 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.096763 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:48:23.096996 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.096841 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:48:23.096996 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.096943 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:48:23.097232 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.097011 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:48:23.097299 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.097262 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:48:23.097375 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.097340 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:48:23.097495 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.097471 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:48:23.097900 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.097872 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:48:23.098424 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.098401 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:48:23.098969 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.098952 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe44269f-1286-4d41-a2bd-65a10d29a9df-config-out" (OuterVolumeSpecName: "config-out") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:48:23.099259 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.099244 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-config" (OuterVolumeSpecName: "config") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:48:23.099348 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.099335 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:48:23.109895 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.109868 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-web-config" (OuterVolumeSpecName: "web-config") pod "fe44269f-1286-4d41-a2bd-65a10d29a9df" (UID: "fe44269f-1286-4d41-a2bd-65a10d29a9df"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:48:23.195842 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195755 2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.195842 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195789 2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-web-config\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.195842 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195799 2568 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-configmap-metrics-client-ca\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.195842 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195810 2568 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-config\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.195842 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195819 2568 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.195842 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195827 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.195842 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195836 2568 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.195842 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195846 2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fe44269f-1286-4d41-a2bd-65a10d29a9df-config-out\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.196244 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195857 2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.196244 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195865 2568 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-grpc-tls\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.196244 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195873 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.196244 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195881 2568 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.196244 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195889 2568 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-kube-rbac-proxy\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.196244 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195897 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-db\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.196244 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195905 2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fe44269f-1286-4d41-a2bd-65a10d29a9df-tls-assets\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.196244 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195914 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss95f\" (UniqueName: \"kubernetes.io/projected/fe44269f-1286-4d41-a2bd-65a10d29a9df-kube-api-access-ss95f\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.196244 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195924 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fe44269f-1286-4d41-a2bd-65a10d29a9df-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.196244 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195932 2568 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-metrics-client-certs\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.196244 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.195941 2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fe44269f-1286-4d41-a2bd-65a10d29a9df-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:23.236451 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.236419 2568 generic.go:356] "Generic (PLEG): container finished" podID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerID="73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c" exitCode=0 Jul 08 19:48:23.236451 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.236441 2568 generic.go:356] "Generic (PLEG): container finished" podID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerID="4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7" exitCode=0 Jul 08 19:48:23.236451 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.236448 2568 generic.go:356] "Generic (PLEG): container finished" podID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerID="d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744" exitCode=0 Jul 08 19:48:23.236451 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.236454 2568 generic.go:356] "Generic (PLEG): container finished" podID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerID="382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd" exitCode=0 Jul 08 19:48:23.236451 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.236460 2568 generic.go:356] "Generic (PLEG): container finished" podID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerID="8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee" exitCode=0 Jul 08 19:48:23.236981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.236466 2568 generic.go:356] "Generic (PLEG): container finished" podID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerID="8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d" exitCode=0 Jul 08 19:48:23.236981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.236494 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.236981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.236494 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe44269f-1286-4d41-a2bd-65a10d29a9df","Type":"ContainerDied","Data":"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c"} Jul 08 19:48:23.236981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.236621 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe44269f-1286-4d41-a2bd-65a10d29a9df","Type":"ContainerDied","Data":"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7"} Jul 08 19:48:23.236981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.236648 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe44269f-1286-4d41-a2bd-65a10d29a9df","Type":"ContainerDied","Data":"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744"} Jul 08 19:48:23.236981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.236676 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe44269f-1286-4d41-a2bd-65a10d29a9df","Type":"ContainerDied","Data":"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd"} Jul 08 19:48:23.236981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.236696 2568 scope.go:117] "RemoveContainer" containerID="73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c" Jul 08 19:48:23.236981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.236700 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe44269f-1286-4d41-a2bd-65a10d29a9df","Type":"ContainerDied","Data":"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee"} Jul 08 19:48:23.236981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.236831 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe44269f-1286-4d41-a2bd-65a10d29a9df","Type":"ContainerDied","Data":"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d"} Jul 08 19:48:23.236981 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.236875 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe44269f-1286-4d41-a2bd-65a10d29a9df","Type":"ContainerDied","Data":"247555fee2fb6f830eba6af7cc1a42ee117dd33b4d9d0a72838d12607c35fc4e"} Jul 08 19:48:23.248072 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.246246 2568 scope.go:117] "RemoveContainer" containerID="4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7" Jul 08 19:48:23.254771 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.254752 2568 scope.go:117] "RemoveContainer" containerID="d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744" Jul 08 19:48:23.262008 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.261984 2568 scope.go:117] "RemoveContainer" containerID="382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd" Jul 08 19:48:23.265838 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.265812 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 19:48:23.271099 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.271078 2568 scope.go:117] "RemoveContainer" containerID="8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee" Jul 08 19:48:23.271332 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.271314 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 19:48:23.278168 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.278148 2568 scope.go:117] "RemoveContainer" containerID="8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d" Jul 08 19:48:23.285643 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.285622 2568 scope.go:117] "RemoveContainer" containerID="7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841" Jul 08 19:48:23.292987 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.292972 2568 scope.go:117] "RemoveContainer" containerID="73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c" Jul 08 19:48:23.293304 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:48:23.293282 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c\": container with ID starting with 73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c not found: ID does not exist" containerID="73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c" Jul 08 19:48:23.293393 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.293319 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c"} err="failed to get container status \"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c\": rpc error: code = NotFound desc = could not find container \"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c\": container with ID starting with 73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c not found: ID does not exist" Jul 08 19:48:23.293393 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.293340 2568 scope.go:117] "RemoveContainer" containerID="4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7" Jul 08 19:48:23.293620 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:48:23.293602 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7\": container with ID starting with 4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7 not found: ID does not exist" containerID="4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7" Jul 08 19:48:23.293656 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.293626 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7"} err="failed to get container status \"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7\": rpc error: code = NotFound desc = could not find container \"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7\": container with ID starting with 4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7 not found: ID does not exist" Jul 08 19:48:23.293656 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.293642 2568 scope.go:117] "RemoveContainer" containerID="d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744" Jul 08 19:48:23.293858 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:48:23.293841 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744\": container with ID starting with d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744 not found: ID does not exist" containerID="d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744" Jul 08 19:48:23.293906 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.293862 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744"} err="failed to get container status \"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744\": rpc error: code = NotFound desc = could not find container \"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744\": container with ID starting with d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744 not found: ID does not exist" Jul 08 19:48:23.293906 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.293877 2568 scope.go:117] "RemoveContainer" containerID="382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd" Jul 08 19:48:23.294541 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:48:23.294516 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd\": container with ID starting with 382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd not found: ID does not exist" containerID="382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd" Jul 08 19:48:23.294763 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.294739 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd"} err="failed to get container status \"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd\": rpc error: code = NotFound desc = could not find container \"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd\": container with ID starting with 382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd not found: ID does not exist" Jul 08 19:48:23.295191 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.294882 2568 scope.go:117] "RemoveContainer" containerID="8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee" Jul 08 19:48:23.297512 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:48:23.297485 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee\": container with ID starting with 8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee not found: ID does not exist" containerID="8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee" Jul 08 19:48:23.297621 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.297519 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee"} err="failed to get container status \"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee\": rpc error: code = NotFound desc = could not find container \"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee\": container with ID starting with 8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee not found: ID does not exist" Jul 08 19:48:23.297621 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.297540 2568 scope.go:117] "RemoveContainer" containerID="8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d" Jul 08 19:48:23.297998 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.297977 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 19:48:23.298224 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:48:23.298206 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d\": container with ID starting with 8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d not found: ID does not exist" containerID="8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d" Jul 08 19:48:23.298305 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298232 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d"} err="failed to get container status \"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d\": rpc error: code = NotFound desc = could not find container \"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d\": container with ID starting with 8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d not found: ID does not exist" Jul 08 19:48:23.298305 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298253 2568 scope.go:117] "RemoveContainer" containerID="7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841" Jul 08 19:48:23.298541 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298514 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="init-config-reloader" Jul 08 19:48:23.298541 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298536 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="init-config-reloader" Jul 08 19:48:23.298689 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298548 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="kube-rbac-proxy" Jul 08 19:48:23.298689 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298554 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="kube-rbac-proxy" Jul 08 19:48:23.298689 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298564 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="config-reloader" Jul 08 19:48:23.298689 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298590 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="config-reloader" Jul 08 19:48:23.298689 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298601 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="kube-rbac-proxy-thanos" Jul 08 19:48:23.298689 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298610 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="kube-rbac-proxy-thanos" Jul 08 19:48:23.298689 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298616 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="prometheus" Jul 08 19:48:23.298689 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298622 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="prometheus" Jul 08 19:48:23.298689 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298629 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="kube-rbac-proxy-web" Jul 08 19:48:23.298689 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298634 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="kube-rbac-proxy-web" Jul 08 19:48:23.298689 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298639 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="thanos-sidecar" Jul 08 19:48:23.298689 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298644 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="thanos-sidecar" Jul 08 19:48:23.299285 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298735 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="kube-rbac-proxy-thanos" Jul 08 19:48:23.299285 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:48:23.298730 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841\": container with ID starting with 7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841 not found: ID does not exist" containerID="7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841" Jul 08 19:48:23.299285 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298747 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="config-reloader" Jul 08 19:48:23.299285 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298760 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="kube-rbac-proxy-web" Jul 08 19:48:23.299285 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298768 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="prometheus" Jul 08 19:48:23.299285 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298777 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="thanos-sidecar" Jul 08 19:48:23.299285 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298785 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" containerName="kube-rbac-proxy" Jul 08 19:48:23.299285 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298759 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841"} err="failed to get container status \"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841\": rpc error: code = NotFound desc = could not find container \"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841\": container with ID starting with 7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841 not found: ID does not exist" Jul 08 19:48:23.299285 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.298951 2568 scope.go:117] "RemoveContainer" containerID="73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c" Jul 08 19:48:23.299697 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.299289 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c"} err="failed to get container status \"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c\": rpc error: code = NotFound desc = could not find container \"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c\": container with ID starting with 73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c not found: ID does not exist" Jul 08 19:48:23.299697 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.299311 2568 scope.go:117] "RemoveContainer" containerID="4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7" Jul 08 19:48:23.299697 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.299670 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7"} err="failed to get container status \"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7\": rpc error: code = NotFound desc = could not find container \"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7\": container with ID starting with 4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7 not found: ID does not exist" Jul 08 19:48:23.299697 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.299693 2568 scope.go:117] "RemoveContainer" containerID="d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744" Jul 08 19:48:23.299989 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.299966 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744"} err="failed to get container status \"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744\": rpc error: code = NotFound desc = could not find container \"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744\": container with ID starting with d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744 not found: ID does not exist" Jul 08 19:48:23.300059 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.299991 2568 scope.go:117] "RemoveContainer" containerID="382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd" Jul 08 19:48:23.300289 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.300260 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd"} err="failed to get container status \"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd\": rpc error: code = NotFound desc = could not find container \"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd\": container with ID starting with 382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd not found: ID does not exist" Jul 08 19:48:23.300289 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.300283 2568 scope.go:117] "RemoveContainer" containerID="8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee" Jul 08 19:48:23.300542 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.300524 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee"} err="failed to get container status \"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee\": rpc error: code = NotFound desc = could not find container \"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee\": container with ID starting with 8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee not found: ID does not exist" Jul 08 19:48:23.300619 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.300543 2568 scope.go:117] "RemoveContainer" containerID="8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d" Jul 08 19:48:23.300779 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.300760 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d"} err="failed to get container status \"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d\": rpc error: code = NotFound desc = could not find container \"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d\": container with ID starting with 8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d not found: ID does not exist" Jul 08 19:48:23.300833 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.300780 2568 scope.go:117] "RemoveContainer" containerID="7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841" Jul 08 19:48:23.300995 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.300974 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841"} err="failed to get container status \"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841\": rpc error: code = NotFound desc = could not find container \"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841\": container with ID starting with 7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841 not found: ID does not exist" Jul 08 19:48:23.300995 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.300994 2568 scope.go:117] "RemoveContainer" containerID="73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c" Jul 08 19:48:23.301263 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.301246 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c"} err="failed to get container status \"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c\": rpc error: code = NotFound desc = could not find container \"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c\": container with ID starting with 73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c not found: ID does not exist" Jul 08 19:48:23.301317 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.301264 2568 scope.go:117] "RemoveContainer" containerID="4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7" Jul 08 19:48:23.301479 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.301462 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7"} err="failed to get container status \"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7\": rpc error: code = NotFound desc = could not find container \"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7\": container with ID starting with 4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7 not found: ID does not exist" Jul 08 19:48:23.301533 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.301481 2568 scope.go:117] "RemoveContainer" containerID="d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744" Jul 08 19:48:23.301695 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.301678 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744"} err="failed to get container status \"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744\": rpc error: code = NotFound desc = could not find container \"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744\": container with ID starting with d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744 not found: ID does not exist" Jul 08 19:48:23.301740 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.301696 2568 scope.go:117] "RemoveContainer" containerID="382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd" Jul 08 19:48:23.301984 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.301967 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd"} err="failed to get container status \"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd\": rpc error: code = NotFound desc = could not find container \"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd\": container with ID starting with 382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd not found: ID does not exist" Jul 08 19:48:23.301984 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.301983 2568 scope.go:117] "RemoveContainer" containerID="8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee" Jul 08 19:48:23.302271 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.302255 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee"} err="failed to get container status \"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee\": rpc error: code = NotFound desc = could not find container \"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee\": container with ID starting with 8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee not found: ID does not exist" Jul 08 19:48:23.302310 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.302271 2568 scope.go:117] "RemoveContainer" containerID="8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d" Jul 08 19:48:23.302482 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.302465 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d"} err="failed to get container status \"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d\": rpc error: code = NotFound desc = could not find container \"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d\": container with ID starting with 8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d not found: ID does not exist" Jul 08 19:48:23.302523 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.302484 2568 scope.go:117] "RemoveContainer" containerID="7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841" Jul 08 19:48:23.302665 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.302649 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841"} err="failed to get container status \"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841\": rpc error: code = NotFound desc = could not find container \"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841\": container with ID starting with 7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841 not found: ID does not exist" Jul 08 19:48:23.302708 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.302665 2568 scope.go:117] "RemoveContainer" containerID="73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c" Jul 08 19:48:23.302861 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.302846 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c"} err="failed to get container status \"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c\": rpc error: code = NotFound desc = could not find container \"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c\": container with ID starting with 73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c not found: ID does not exist" Jul 08 19:48:23.302895 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.302861 2568 scope.go:117] "RemoveContainer" containerID="4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7" Jul 08 19:48:23.303194 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.303171 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7"} err="failed to get container status \"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7\": rpc error: code = NotFound desc = could not find container \"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7\": container with ID starting with 4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7 not found: ID does not exist" Jul 08 19:48:23.303194 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.303194 2568 scope.go:117] "RemoveContainer" containerID="d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744" Jul 08 19:48:23.303464 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.303442 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744"} err="failed to get container status \"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744\": rpc error: code = NotFound desc = could not find container \"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744\": container with ID starting with d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744 not found: ID does not exist" Jul 08 19:48:23.303464 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.303464 2568 scope.go:117] "RemoveContainer" containerID="382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd" Jul 08 19:48:23.305457 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.303679 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.305898 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.305768 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd"} err="failed to get container status \"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd\": rpc error: code = NotFound desc = could not find container \"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd\": container with ID starting with 382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd not found: ID does not exist" Jul 08 19:48:23.305898 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.305810 2568 scope.go:117] "RemoveContainer" containerID="8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee" Jul 08 19:48:23.306133 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.306106 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee"} err="failed to get container status \"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee\": rpc error: code = NotFound desc = could not find container \"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee\": container with ID starting with 8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee not found: ID does not exist" Jul 08 19:48:23.306133 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.306127 2568 scope.go:117] "RemoveContainer" containerID="8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d" Jul 08 19:48:23.306356 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.306339 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d"} err="failed to get container status \"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d\": rpc error: code = NotFound desc = could not find container \"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d\": container with ID starting with 8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d not found: ID does not exist" Jul 08 19:48:23.306356 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.306355 2568 scope.go:117] "RemoveContainer" containerID="7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841" Jul 08 19:48:23.306535 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.306518 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841"} err="failed to get container status \"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841\": rpc error: code = NotFound desc = could not find container \"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841\": container with ID starting with 7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841 not found: ID does not exist" Jul 08 19:48:23.306535 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.306534 2568 scope.go:117] "RemoveContainer" containerID="73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c" Jul 08 19:48:23.306722 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.306705 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c"} err="failed to get container status \"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c\": rpc error: code = NotFound desc = could not find container \"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c\": container with ID starting with 73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c not found: ID does not exist" Jul 08 19:48:23.306769 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.306722 2568 scope.go:117] "RemoveContainer" containerID="4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7" Jul 08 19:48:23.306963 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.306941 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7"} err="failed to get container status \"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7\": rpc error: code = NotFound desc = could not find container \"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7\": container with ID starting with 4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7 not found: ID does not exist" Jul 08 19:48:23.306963 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.306963 2568 scope.go:117] "RemoveContainer" containerID="d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744" Jul 08 19:48:23.307206 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.307187 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744"} err="failed to get container status \"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744\": rpc error: code = NotFound desc = could not find container \"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744\": container with ID starting with d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744 not found: ID does not exist" Jul 08 19:48:23.307313 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.307207 2568 scope.go:117] "RemoveContainer" containerID="382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd" Jul 08 19:48:23.307430 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.307415 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd"} err="failed to get container status \"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd\": rpc error: code = NotFound desc = could not find container \"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd\": container with ID starting with 382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd not found: ID does not exist" Jul 08 19:48:23.307430 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.307430 2568 scope.go:117] "RemoveContainer" containerID="8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee" Jul 08 19:48:23.307629 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.307614 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee"} err="failed to get container status \"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee\": rpc error: code = NotFound desc = could not find container \"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee\": container with ID starting with 8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee not found: ID does not exist" Jul 08 19:48:23.307629 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.307628 2568 scope.go:117] "RemoveContainer" containerID="8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d" Jul 08 19:48:23.307833 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.307817 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d"} err="failed to get container status \"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d\": rpc error: code = NotFound desc = could not find container \"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d\": container with ID starting with 8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d not found: ID does not exist" Jul 08 19:48:23.307885 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.307833 2568 scope.go:117] "RemoveContainer" containerID="7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841" Jul 08 19:48:23.308094 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.308068 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841"} err="failed to get container status \"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841\": rpc error: code = NotFound desc = could not find container \"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841\": container with ID starting with 7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841 not found: ID does not exist" Jul 08 19:48:23.308094 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.308094 2568 scope.go:117] "RemoveContainer" containerID="73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c" Jul 08 19:48:23.308317 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.308294 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c"} err="failed to get container status \"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c\": rpc error: code = NotFound desc = could not find container \"73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c\": container with ID starting with 73b264b6be6e8a58ab0732ba452f00e0b52e96ac813f92c1bd6bd98fe716854c not found: ID does not exist" Jul 08 19:48:23.308364 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.308318 2568 scope.go:117] "RemoveContainer" containerID="4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7" Jul 08 19:48:23.308558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.308531 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7"} err="failed to get container status \"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7\": rpc error: code = NotFound desc = could not find container \"4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7\": container with ID starting with 4a092d714855bb153f44a6b9a22c3a8f084fd3d7b902286f257356ad9ad2a0f7 not found: ID does not exist" Jul 08 19:48:23.308558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.308556 2568 scope.go:117] "RemoveContainer" containerID="d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744" Jul 08 19:48:23.308839 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.308807 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744"} err="failed to get container status \"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744\": rpc error: code = NotFound desc = could not find container \"d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744\": container with ID starting with d2ff3fd9c8b82dea18a44ec371b5b37b0452c96340ed07c99c69243c85811744 not found: ID does not exist" Jul 08 19:48:23.308839 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.308840 2568 scope.go:117] "RemoveContainer" containerID="382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd" Jul 08 19:48:23.309009 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.308992 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 19:48:23.309133 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.309118 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 19:48:23.309133 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.309131 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 19:48:23.309249 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.309165 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 19:48:23.309249 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.309189 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 19:48:23.309350 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.309248 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-n9spx\"" Jul 08 19:48:23.309350 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.309304 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 19:48:23.309350 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.309316 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 19:48:23.309350 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.309338 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 19:48:23.309589 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.309369 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 19:48:23.309589 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.309461 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 19:48:23.309688 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.309632 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 19:48:23.309688 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.309655 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd"} err="failed to get container status \"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd\": rpc error: code = NotFound desc = could not find container \"382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd\": container with ID starting with 382b39a41b8587f4d58b507ecdcbbe295384ad53e0576b455e2f80853884d0bd not found: ID does not exist" Jul 08 19:48:23.309688 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.309677 2568 scope.go:117] "RemoveContainer" containerID="8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee" Jul 08 19:48:23.309809 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.309699 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-12ed5j18ulim6\"" Jul 08 19:48:23.309809 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.309722 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 19:48:23.309953 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.309929 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee"} err="failed to get container status \"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee\": rpc error: code = NotFound desc = could not find container \"8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee\": container with ID starting with 8ed6a157a3fe138f8f3599d554910e7ebeecb1287a9d8c8d0195940b1f6d4bee not found: ID does not exist" Jul 08 19:48:23.310011 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.309956 2568 scope.go:117] "RemoveContainer" containerID="8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d" Jul 08 19:48:23.310251 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.310230 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d"} err="failed to get container status \"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d\": rpc error: code = NotFound desc = could not find container \"8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d\": container with ID starting with 8227d41968fa45d5587046afddf9340292a65bd749dac8a21e0ede93adf28f0d not found: ID does not exist" Jul 08 19:48:23.310324 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.310253 2568 scope.go:117] "RemoveContainer" containerID="7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841" Jul 08 19:48:23.310607 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.310566 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841"} err="failed to get container status \"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841\": rpc error: code = NotFound desc = could not find container \"7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841\": container with ID starting with 7840a9e06884e61d3dae2e04bb4c673f530933fafb9951ea8cd5b805c1819841 not found: ID does not exist" Jul 08 19:48:23.312800 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.312782 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 19:48:23.316647 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.316628 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 19:48:23.327801 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.326846 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 19:48:23.398907 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.398874 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-config\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.398907 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.398907 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.399269 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.398948 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.399269 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.399072 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.399269 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.399120 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.399269 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.399149 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.399269 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.399181 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf22g\" (UniqueName: \"kubernetes.io/projected/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-kube-api-access-cf22g\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.399516 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.399280 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.399516 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.399322 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.399516 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.399363 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.399516 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.399393 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.399516 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.399431 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-config-out\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.399743 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.399603 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.399743 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.399649 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.399743 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.399690 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.399890 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.399763 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.399890 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.399787 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-web-config\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.399890 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.399821 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.399890 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.399850 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.400100 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.399911 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.500703 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.500618 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.500703 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.500652 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.500703 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.500679 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-config-out\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.500987 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.500717 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.500987 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.500745 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.500987 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.500881 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.500987 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.500940 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.500987 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.500970 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-web-config\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.501275 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.500998 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.501275 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.501025 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.501275 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.501090 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.501275 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.501101 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.501275 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.501144 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-config\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.501275 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.501177 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.501275 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.501220 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.501275 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.501248 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.501275 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.501272 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.503602 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.501294 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.503602 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.501317 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf22g\" (UniqueName: \"kubernetes.io/projected/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-kube-api-access-cf22g\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.503602 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.501370 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.503602 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.501403 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.503602 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.501523 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.503602 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.501602 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.503602 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.501710 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.503602 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.501861 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.503602 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.502424 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.503602 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.502940 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.503602 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.503591 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-config-out\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.504182 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.504018 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.505054 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.504973 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.505054 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.504986 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-config\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.505247 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.505221 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.505440 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.505416 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.505653 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.505632 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.506148 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.506125 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.506148 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.506137 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.506613 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.506590 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.506613 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.506605 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.506923 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.506905 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-web-config\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.511807 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.511787 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf22g\" (UniqueName: \"kubernetes.io/projected/40c14187-eb76-4a4d-944e-a0f0f7b05bc4-kube-api-access-cf22g\") pod \"prometheus-k8s-0\" (UID: \"40c14187-eb76-4a4d-944e-a0f0f7b05bc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.514954 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.514935 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:23.611932 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.611893 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:23.616010 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.615980 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:23.713940 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.713900 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:23.761901 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:23.761875 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 19:48:23.763706 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:48:23.763685 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c14187_eb76_4a4d_944e_a0f0f7b05bc4.slice/crio-1f6b6d044c44a5bb6a8bf1d0513b7cc786dccb1f834073a9b61e2dce9430d9cf WatchSource:0}: Error finding container 1f6b6d044c44a5bb6a8bf1d0513b7cc786dccb1f834073a9b61e2dce9430d9cf: Status 404 returned error can't find the container with id 1f6b6d044c44a5bb6a8bf1d0513b7cc786dccb1f834073a9b61e2dce9430d9cf Jul 08 19:48:24.209792 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:24.209761 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:24.244910 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:24.244881 2568 generic.go:356] "Generic (PLEG): container finished" podID="40c14187-eb76-4a4d-944e-a0f0f7b05bc4" containerID="a8c5f9c57edd36d1f8269c16ac7f44f6552697ab7a543163fdce9a4f0e596660" exitCode=0 Jul 08 19:48:24.245361 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:24.244977 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40c14187-eb76-4a4d-944e-a0f0f7b05bc4","Type":"ContainerDied","Data":"a8c5f9c57edd36d1f8269c16ac7f44f6552697ab7a543163fdce9a4f0e596660"} Jul 08 19:48:24.245361 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:24.245069 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40c14187-eb76-4a4d-944e-a0f0f7b05bc4","Type":"ContainerStarted","Data":"1f6b6d044c44a5bb6a8bf1d0513b7cc786dccb1f834073a9b61e2dce9430d9cf"} Jul 08 19:48:24.276367 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:24.276334 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:24.831537 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:24.831506 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe44269f-1286-4d41-a2bd-65a10d29a9df" path="/var/lib/kubelet/pods/fe44269f-1286-4d41-a2bd-65a10d29a9df/volumes" Jul 08 19:48:25.253447 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:25.253359 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40c14187-eb76-4a4d-944e-a0f0f7b05bc4","Type":"ContainerStarted","Data":"3edc766f54a0b36cd28a1eb6e9a228084b67d5b0174fd3aa99e1c57915c824a3"} Jul 08 19:48:25.253447 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:25.253437 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40c14187-eb76-4a4d-944e-a0f0f7b05bc4","Type":"ContainerStarted","Data":"5131d87495f577817ba966d24360bbe4900742de8720fa6fbd1f0ebc4b6563f5"} Jul 08 19:48:25.253833 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:25.253481 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40c14187-eb76-4a4d-944e-a0f0f7b05bc4","Type":"ContainerStarted","Data":"09ae62f13a112952bce53e09fb239e081f1fe9e6e5f3233b00a09b9b240a72ef"} Jul 08 19:48:25.253833 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:25.253519 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40c14187-eb76-4a4d-944e-a0f0f7b05bc4","Type":"ContainerStarted","Data":"bb891d4dae4449bf7fe4a695c8799654103ccc14ab8744cad500eab8764d4e8a"} Jul 08 19:48:25.253833 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:25.253552 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40c14187-eb76-4a4d-944e-a0f0f7b05bc4","Type":"ContainerStarted","Data":"67c85d4577d63ee3b10179e1538812b346db5801a1e6476c7d6f297c0ed0a512"} Jul 08 19:48:25.253833 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:25.253587 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40c14187-eb76-4a4d-944e-a0f0f7b05bc4","Type":"ContainerStarted","Data":"638aecaec401158ef160b1290f9167ba8df5c368ec7a677aa636bbe5f4dc2517"} Jul 08 19:48:25.284940 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:25.284881 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.264884372 podStartE2EDuration="2.284863188s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.978816ms" isStatefulPod=false podCreationTimestamp="2026-07-08 19:48:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:48:25.282019358 +0000 UTC m=+386.975328872" watchObservedRunningTime="2026-07-08 19:48:25.284863188 +0000 UTC m=+386.978172673" Jul 08 19:48:26.102119 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:26.102086 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:26.157703 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:26.157652 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:28.616930 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:28.616879 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:48:33.102744 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:33.102710 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:33.200460 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:33.200418 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:34.180200 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:34.180167 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8cfc6759c-shwbv"] Jul 08 19:48:34.279306 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:34.279270 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:34.331466 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:34.331436 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:34.389382 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:34.389350 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:35.556525 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:35.556490 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:51.923330 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:51.923291 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:51.978310 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:51.978275 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:48:59.201443 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.201385 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-8cfc6759c-shwbv" podUID="75725cee-3256-48ee-b4b1-0370ab088770" containerName="console" containerID="cri-o://dd46eb83ed14a8b08e866920930104f7f9bf5e81d98b65edf7df5d24040401cc" gracePeriod=15 Jul 08 19:48:59.408825 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.408790 2568 generic.go:356] "Generic (PLEG): container finished" podID="75725cee-3256-48ee-b4b1-0370ab088770" containerID="dd46eb83ed14a8b08e866920930104f7f9bf5e81d98b65edf7df5d24040401cc" exitCode=0 Jul 08 19:48:59.408977 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.408847 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8cfc6759c-shwbv" event={"ID":"75725cee-3256-48ee-b4b1-0370ab088770","Type":"ContainerDied","Data":"dd46eb83ed14a8b08e866920930104f7f9bf5e81d98b65edf7df5d24040401cc"} Jul 08 19:48:59.443649 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.443622 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:48:59.513846 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.513762 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75725cee-3256-48ee-b4b1-0370ab088770-console-oauth-config\") pod \"75725cee-3256-48ee-b4b1-0370ab088770\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " Jul 08 19:48:59.513846 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.513798 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cxx4\" (UniqueName: \"kubernetes.io/projected/75725cee-3256-48ee-b4b1-0370ab088770-kube-api-access-7cxx4\") pod \"75725cee-3256-48ee-b4b1-0370ab088770\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " Jul 08 19:48:59.513846 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.513836 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-service-ca\") pod \"75725cee-3256-48ee-b4b1-0370ab088770\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " Jul 08 19:48:59.514160 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.513858 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-trusted-ca-bundle\") pod \"75725cee-3256-48ee-b4b1-0370ab088770\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " Jul 08 19:48:59.514160 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.514000 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-console-config\") pod \"75725cee-3256-48ee-b4b1-0370ab088770\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " Jul 08 19:48:59.514160 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.514101 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75725cee-3256-48ee-b4b1-0370ab088770-console-serving-cert\") pod \"75725cee-3256-48ee-b4b1-0370ab088770\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " Jul 08 19:48:59.514160 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.514140 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-oauth-serving-cert\") pod \"75725cee-3256-48ee-b4b1-0370ab088770\" (UID: \"75725cee-3256-48ee-b4b1-0370ab088770\") " Jul 08 19:48:59.514368 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.514294 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "75725cee-3256-48ee-b4b1-0370ab088770" (UID: "75725cee-3256-48ee-b4b1-0370ab088770"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:48:59.514368 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.514332 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-service-ca" (OuterVolumeSpecName: "service-ca") pod "75725cee-3256-48ee-b4b1-0370ab088770" (UID: "75725cee-3256-48ee-b4b1-0370ab088770"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:48:59.514475 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.514403 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-console-config" (OuterVolumeSpecName: "console-config") pod "75725cee-3256-48ee-b4b1-0370ab088770" (UID: "75725cee-3256-48ee-b4b1-0370ab088770"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:48:59.514610 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.514592 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "75725cee-3256-48ee-b4b1-0370ab088770" (UID: "75725cee-3256-48ee-b4b1-0370ab088770"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:48:59.514682 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.514612 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-service-ca\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:59.514682 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.514632 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-trusted-ca-bundle\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:59.514682 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.514641 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-console-config\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:59.516159 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.516138 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75725cee-3256-48ee-b4b1-0370ab088770-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "75725cee-3256-48ee-b4b1-0370ab088770" (UID: "75725cee-3256-48ee-b4b1-0370ab088770"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:48:59.516504 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.516476 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75725cee-3256-48ee-b4b1-0370ab088770-kube-api-access-7cxx4" (OuterVolumeSpecName: "kube-api-access-7cxx4") pod "75725cee-3256-48ee-b4b1-0370ab088770" (UID: "75725cee-3256-48ee-b4b1-0370ab088770"). InnerVolumeSpecName "kube-api-access-7cxx4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:48:59.516504 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.516481 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75725cee-3256-48ee-b4b1-0370ab088770-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "75725cee-3256-48ee-b4b1-0370ab088770" (UID: "75725cee-3256-48ee-b4b1-0370ab088770"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:48:59.615362 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.615312 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75725cee-3256-48ee-b4b1-0370ab088770-console-serving-cert\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:59.615362 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.615355 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75725cee-3256-48ee-b4b1-0370ab088770-oauth-serving-cert\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:59.615362 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.615366 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75725cee-3256-48ee-b4b1-0370ab088770-console-oauth-config\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:48:59.615362 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:48:59.615375 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cxx4\" (UniqueName: \"kubernetes.io/projected/75725cee-3256-48ee-b4b1-0370ab088770-kube-api-access-7cxx4\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:49:00.414913 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:00.414884 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8cfc6759c-shwbv" Jul 08 19:49:00.414913 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:00.414894 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8cfc6759c-shwbv" event={"ID":"75725cee-3256-48ee-b4b1-0370ab088770","Type":"ContainerDied","Data":"8b1c319396f1a5f349e94d9ec58ee05d278721a39df733be0b75b75fc400d9d6"} Jul 08 19:49:00.415415 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:00.414956 2568 scope.go:117] "RemoveContainer" containerID="dd46eb83ed14a8b08e866920930104f7f9bf5e81d98b65edf7df5d24040401cc" Jul 08 19:49:00.436880 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:00.436853 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8cfc6759c-shwbv"] Jul 08 19:49:00.440283 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:00.440258 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8cfc6759c-shwbv"] Jul 08 19:49:00.831737 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:00.831697 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75725cee-3256-48ee-b4b1-0370ab088770" path="/var/lib/kubelet/pods/75725cee-3256-48ee-b4b1-0370ab088770/volumes" Jul 08 19:49:03.860065 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:03.860005 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:03.958612 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:03.958577 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:23.617087 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:23.616971 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:49:23.632724 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:23.632695 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:49:24.536019 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:24.535991 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 19:49:30.556200 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:30.556165 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:31.377246 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:31.377208 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:34.399415 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:34.399378 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:34.499426 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:34.499390 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:39.003108 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:39.003068 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:39.103397 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:39.103365 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:42.407349 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:42.407317 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:42.511156 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:42.511123 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:42.805734 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:42.805606 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:42.900607 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:42.900576 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:44.803092 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:44.803022 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:45.007474 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:45.007430 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:52.021905 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:52.021868 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:53.120717 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:53.120683 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-zqx5l"] Jul 08 19:49:53.121100 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:53.121065 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75725cee-3256-48ee-b4b1-0370ab088770" containerName="console" Jul 08 19:49:53.121100 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:53.121081 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="75725cee-3256-48ee-b4b1-0370ab088770" containerName="console" Jul 08 19:49:53.121180 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:53.121163 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="75725cee-3256-48ee-b4b1-0370ab088770" containerName="console" Jul 08 19:49:53.124301 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:53.124282 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-zqx5l" Jul 08 19:49:53.127269 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:53.127236 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 19:49:53.127425 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:53.127278 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-rbcp9\"" Jul 08 19:49:53.128720 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:53.128705 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 19:49:53.133990 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:53.133969 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-zqx5l"] Jul 08 19:49:53.188947 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:53.188913 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fb828f6-3c9c-4265-b440-5a4c4ca1d958-bound-sa-token\") pod \"cert-manager-55b4d954d5-zqx5l\" (UID: \"6fb828f6-3c9c-4265-b440-5a4c4ca1d958\") " pod="cert-manager/cert-manager-55b4d954d5-zqx5l" Jul 08 19:49:53.189121 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:53.188981 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6542k\" (UniqueName: \"kubernetes.io/projected/6fb828f6-3c9c-4265-b440-5a4c4ca1d958-kube-api-access-6542k\") pod \"cert-manager-55b4d954d5-zqx5l\" (UID: \"6fb828f6-3c9c-4265-b440-5a4c4ca1d958\") " pod="cert-manager/cert-manager-55b4d954d5-zqx5l" Jul 08 19:49:53.289700 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:53.289660 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6542k\" (UniqueName: \"kubernetes.io/projected/6fb828f6-3c9c-4265-b440-5a4c4ca1d958-kube-api-access-6542k\") pod \"cert-manager-55b4d954d5-zqx5l\" (UID: \"6fb828f6-3c9c-4265-b440-5a4c4ca1d958\") " pod="cert-manager/cert-manager-55b4d954d5-zqx5l" Jul 08 19:49:53.289864 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:53.289724 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fb828f6-3c9c-4265-b440-5a4c4ca1d958-bound-sa-token\") pod \"cert-manager-55b4d954d5-zqx5l\" (UID: \"6fb828f6-3c9c-4265-b440-5a4c4ca1d958\") " pod="cert-manager/cert-manager-55b4d954d5-zqx5l" Jul 08 19:49:53.298597 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:53.298563 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fb828f6-3c9c-4265-b440-5a4c4ca1d958-bound-sa-token\") pod \"cert-manager-55b4d954d5-zqx5l\" (UID: \"6fb828f6-3c9c-4265-b440-5a4c4ca1d958\") " pod="cert-manager/cert-manager-55b4d954d5-zqx5l" Jul 08 19:49:53.298787 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:53.298766 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6542k\" (UniqueName: \"kubernetes.io/projected/6fb828f6-3c9c-4265-b440-5a4c4ca1d958-kube-api-access-6542k\") pod \"cert-manager-55b4d954d5-zqx5l\" (UID: \"6fb828f6-3c9c-4265-b440-5a4c4ca1d958\") " pod="cert-manager/cert-manager-55b4d954d5-zqx5l" Jul 08 19:49:53.453998 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:53.453911 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-zqx5l" Jul 08 19:49:53.588532 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:53.588497 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-zqx5l"] Jul 08 19:49:53.591508 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:49:53.591479 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6fb828f6_3c9c_4265_b440_5a4c4ca1d958.slice/crio-e144af6c6310c742dae8de5691e82365281af7d0372503073d60c01503d569e3 WatchSource:0}: Error finding container e144af6c6310c742dae8de5691e82365281af7d0372503073d60c01503d569e3: Status 404 returned error can't find the container with id e144af6c6310c742dae8de5691e82365281af7d0372503073d60c01503d569e3 Jul 08 19:49:53.653725 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:53.653694 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-zqx5l" event={"ID":"6fb828f6-3c9c-4265-b440-5a4c4ca1d958","Type":"ContainerStarted","Data":"e144af6c6310c742dae8de5691e82365281af7d0372503073d60c01503d569e3"} Jul 08 19:49:54.858836 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:54.858796 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:54.935842 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:54.935808 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:56.302908 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:56.302871 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:56.422289 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:56.422225 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:56.671773 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:56.671739 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-zqx5l" event={"ID":"6fb828f6-3c9c-4265-b440-5a4c4ca1d958","Type":"ContainerStarted","Data":"0435bcffb99c3f0207ff3a6b8ede93133f856367551d1beec9c7dc44db9ab514"} Jul 08 19:49:56.688577 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:49:56.688502 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-zqx5l" podStartSLOduration=0.849477313 podStartE2EDuration="3.688482795s" totalImagesPullingTime="2.839005482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:49:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:49:56.688423392 +0000 UTC m=+478.381732873" watchObservedRunningTime="2026-07-08 19:49:56.688482795 +0000 UTC m=+478.381792263" Jul 08 19:50:02.228478 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:02.228444 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:02.340414 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:02.340378 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:04.475362 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:04.475334 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:17.071265 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:17.071230 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:17.173585 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:17.173557 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:17.657965 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:17.657928 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:17.935078 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:17.934980 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:18.030922 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.030890 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:18.083851 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.083820 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4"] Jul 08 19:50:18.088499 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.088481 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:18.091094 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.091058 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/706248ea-19ee-4702-b43f-957e93191854-manifests\") pod \"opendatahub-operator-controller-manager-795d769c5f-p5wt4\" (UID: \"706248ea-19ee-4702-b43f-957e93191854\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:18.091290 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.091111 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/706248ea-19ee-4702-b43f-957e93191854-webhook-cert\") pod \"opendatahub-operator-controller-manager-795d769c5f-p5wt4\" (UID: \"706248ea-19ee-4702-b43f-957e93191854\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:18.091290 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.091174 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/706248ea-19ee-4702-b43f-957e93191854-tmp\") pod \"opendatahub-operator-controller-manager-795d769c5f-p5wt4\" (UID: \"706248ea-19ee-4702-b43f-957e93191854\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:18.091290 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.091218 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phps7\" (UniqueName: \"kubernetes.io/projected/706248ea-19ee-4702-b43f-957e93191854-kube-api-access-phps7\") pod \"opendatahub-operator-controller-manager-795d769c5f-p5wt4\" (UID: \"706248ea-19ee-4702-b43f-957e93191854\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:18.091290 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.091271 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/706248ea-19ee-4702-b43f-957e93191854-apiservice-cert\") pod \"opendatahub-operator-controller-manager-795d769c5f-p5wt4\" (UID: \"706248ea-19ee-4702-b43f-957e93191854\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:18.091753 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.091731 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 19:50:18.091859 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.091775 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 19:50:18.091859 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.091828 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vt2rp\"" Jul 08 19:50:18.092073 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.091926 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 19:50:18.092488 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.092464 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 19:50:18.108308 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.108283 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4"] Jul 08 19:50:18.191819 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.191734 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/706248ea-19ee-4702-b43f-957e93191854-apiservice-cert\") pod \"opendatahub-operator-controller-manager-795d769c5f-p5wt4\" (UID: \"706248ea-19ee-4702-b43f-957e93191854\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:18.191819 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.191790 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/706248ea-19ee-4702-b43f-957e93191854-manifests\") pod \"opendatahub-operator-controller-manager-795d769c5f-p5wt4\" (UID: \"706248ea-19ee-4702-b43f-957e93191854\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:18.192014 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.191819 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/706248ea-19ee-4702-b43f-957e93191854-webhook-cert\") pod \"opendatahub-operator-controller-manager-795d769c5f-p5wt4\" (UID: \"706248ea-19ee-4702-b43f-957e93191854\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:18.192014 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.191931 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/706248ea-19ee-4702-b43f-957e93191854-tmp\") pod \"opendatahub-operator-controller-manager-795d769c5f-p5wt4\" (UID: \"706248ea-19ee-4702-b43f-957e93191854\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:18.192014 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.191977 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phps7\" (UniqueName: \"kubernetes.io/projected/706248ea-19ee-4702-b43f-957e93191854-kube-api-access-phps7\") pod \"opendatahub-operator-controller-manager-795d769c5f-p5wt4\" (UID: \"706248ea-19ee-4702-b43f-957e93191854\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:18.192234 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.192216 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/706248ea-19ee-4702-b43f-957e93191854-manifests\") pod \"opendatahub-operator-controller-manager-795d769c5f-p5wt4\" (UID: \"706248ea-19ee-4702-b43f-957e93191854\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:18.194279 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.194240 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/706248ea-19ee-4702-b43f-957e93191854-tmp\") pod \"opendatahub-operator-controller-manager-795d769c5f-p5wt4\" (UID: \"706248ea-19ee-4702-b43f-957e93191854\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:18.194630 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.194609 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/706248ea-19ee-4702-b43f-957e93191854-webhook-cert\") pod \"opendatahub-operator-controller-manager-795d769c5f-p5wt4\" (UID: \"706248ea-19ee-4702-b43f-957e93191854\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:18.194676 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.194609 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/706248ea-19ee-4702-b43f-957e93191854-apiservice-cert\") pod \"opendatahub-operator-controller-manager-795d769c5f-p5wt4\" (UID: \"706248ea-19ee-4702-b43f-957e93191854\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:18.204449 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.204421 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phps7\" (UniqueName: \"kubernetes.io/projected/706248ea-19ee-4702-b43f-957e93191854-kube-api-access-phps7\") pod \"opendatahub-operator-controller-manager-795d769c5f-p5wt4\" (UID: \"706248ea-19ee-4702-b43f-957e93191854\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:18.215624 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.215596 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:18.314599 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.314566 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:18.400302 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.400273 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:18.419176 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.419149 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:18.508560 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.508527 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:18.540485 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.540450 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4"] Jul 08 19:50:18.770216 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.770127 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" event={"ID":"706248ea-19ee-4702-b43f-957e93191854","Type":"ContainerStarted","Data":"dc58066010e29d66a32349af6de11044b1fe9fa3a8e40c2be30afff21924f8fe"} Jul 08 19:50:18.860889 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.860848 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:18.967265 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:18.967227 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:21.787189 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:21.787089 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" event={"ID":"706248ea-19ee-4702-b43f-957e93191854","Type":"ContainerStarted","Data":"3ab5d43b937ce5c3c506b78746c83d9a23f74ad520e26e788f5842c02c79f8dd"} Jul 08 19:50:22.793085 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:22.793049 2568 generic.go:356] "Generic (PLEG): container finished" podID="706248ea-19ee-4702-b43f-957e93191854" containerID="3ab5d43b937ce5c3c506b78746c83d9a23f74ad520e26e788f5842c02c79f8dd" exitCode=0 Jul 08 19:50:22.793491 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:22.793099 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" event={"ID":"706248ea-19ee-4702-b43f-957e93191854","Type":"ContainerDied","Data":"3ab5d43b937ce5c3c506b78746c83d9a23f74ad520e26e788f5842c02c79f8dd"} Jul 08 19:50:23.799997 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:23.799962 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" event={"ID":"706248ea-19ee-4702-b43f-957e93191854","Type":"ContainerStarted","Data":"3e20412eb59234c06d419de66a2f65723be692d9049ac99730b9d854f48e9a3e"} Jul 08 19:50:23.800443 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:23.800087 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:23.827463 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:23.827410 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" podStartSLOduration=1.427662802 podStartE2EDuration="5.827394064s" totalImagesPullingTime="3.33255961s" totalInitContainerRuntime="1.067171652s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:50:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:50:23.824506685 +0000 UTC m=+505.517816171" watchObservedRunningTime="2026-07-08 19:50:23.827394064 +0000 UTC m=+505.520703550" Jul 08 19:50:25.977127 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:25.977073 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:28.805016 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:28.804980 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:28.908306 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:28.908269 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:29.778455 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:29.778415 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:29.877999 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:29.877969 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:34.807612 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:34.807578 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-p5wt4" Jul 08 19:50:37.391631 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:37.391595 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:37.528585 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:37.528552 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:37.968997 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:37.968963 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:38.216233 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:38.216203 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:38.395837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:38.395801 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:38.490005 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:38.489969 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:38.566126 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:38.566088 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:38.765736 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:38.765647 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:39.181329 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:39.181297 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:39.401410 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:39.401374 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:39.524963 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:39.524867 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:39.611884 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:39.611841 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:40.601131 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:40.601094 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:45.497328 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.497292 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz"] Jul 08 19:50:45.504661 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.504639 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" Jul 08 19:50:45.508055 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.508018 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 19:50:45.508703 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.508687 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:50:45.509811 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.509781 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 19:50:45.509931 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.509845 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 19:50:45.509931 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.509904 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7dkwx\"" Jul 08 19:50:45.510064 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.509928 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 19:50:45.515679 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.515661 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz"] Jul 08 19:50:45.640947 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.640915 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/023f102d-9467-43d7-862d-57e07a82f63a-manager-config\") pod \"lws-controller-manager-7cffbd996-mjwfz\" (UID: \"023f102d-9467-43d7-862d-57e07a82f63a\") " pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" Jul 08 19:50:45.640947 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.640949 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh2db\" (UniqueName: \"kubernetes.io/projected/023f102d-9467-43d7-862d-57e07a82f63a-kube-api-access-fh2db\") pod \"lws-controller-manager-7cffbd996-mjwfz\" (UID: \"023f102d-9467-43d7-862d-57e07a82f63a\") " pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" Jul 08 19:50:45.641175 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.640983 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/023f102d-9467-43d7-862d-57e07a82f63a-cert\") pod \"lws-controller-manager-7cffbd996-mjwfz\" (UID: \"023f102d-9467-43d7-862d-57e07a82f63a\") " pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" Jul 08 19:50:45.641175 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.641082 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/023f102d-9467-43d7-862d-57e07a82f63a-metrics-cert\") pod \"lws-controller-manager-7cffbd996-mjwfz\" (UID: \"023f102d-9467-43d7-862d-57e07a82f63a\") " pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" Jul 08 19:50:45.741560 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.741519 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/023f102d-9467-43d7-862d-57e07a82f63a-cert\") pod \"lws-controller-manager-7cffbd996-mjwfz\" (UID: \"023f102d-9467-43d7-862d-57e07a82f63a\") " pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" Jul 08 19:50:45.741741 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.741600 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/023f102d-9467-43d7-862d-57e07a82f63a-metrics-cert\") pod \"lws-controller-manager-7cffbd996-mjwfz\" (UID: \"023f102d-9467-43d7-862d-57e07a82f63a\") " pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" Jul 08 19:50:45.741741 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.741669 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/023f102d-9467-43d7-862d-57e07a82f63a-manager-config\") pod \"lws-controller-manager-7cffbd996-mjwfz\" (UID: \"023f102d-9467-43d7-862d-57e07a82f63a\") " pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" Jul 08 19:50:45.741741 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.741694 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh2db\" (UniqueName: \"kubernetes.io/projected/023f102d-9467-43d7-862d-57e07a82f63a-kube-api-access-fh2db\") pod \"lws-controller-manager-7cffbd996-mjwfz\" (UID: \"023f102d-9467-43d7-862d-57e07a82f63a\") " pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" Jul 08 19:50:45.742478 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.742455 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/023f102d-9467-43d7-862d-57e07a82f63a-manager-config\") pod \"lws-controller-manager-7cffbd996-mjwfz\" (UID: \"023f102d-9467-43d7-862d-57e07a82f63a\") " pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" Jul 08 19:50:45.744143 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.744123 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/023f102d-9467-43d7-862d-57e07a82f63a-metrics-cert\") pod \"lws-controller-manager-7cffbd996-mjwfz\" (UID: \"023f102d-9467-43d7-862d-57e07a82f63a\") " pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" Jul 08 19:50:45.744296 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.744273 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/023f102d-9467-43d7-862d-57e07a82f63a-cert\") pod \"lws-controller-manager-7cffbd996-mjwfz\" (UID: \"023f102d-9467-43d7-862d-57e07a82f63a\") " pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" Jul 08 19:50:45.752994 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.752933 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh2db\" (UniqueName: \"kubernetes.io/projected/023f102d-9467-43d7-862d-57e07a82f63a-kube-api-access-fh2db\") pod \"lws-controller-manager-7cffbd996-mjwfz\" (UID: \"023f102d-9467-43d7-862d-57e07a82f63a\") " pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" Jul 08 19:50:45.755873 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.755848 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2"] Jul 08 19:50:45.758560 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.758541 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.761785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.761755 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 08 19:50:45.761785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.761772 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jtrmq\"" Jul 08 19:50:45.761785 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.761780 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 19:50:45.762025 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.761834 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 19:50:45.762025 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.761873 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 08 19:50:45.762025 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.761870 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 08 19:50:45.762810 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.762796 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 19:50:45.771450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.771429 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2"] Jul 08 19:50:45.814883 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.814841 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" Jul 08 19:50:45.828603 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.828573 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:45.842796 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.842756 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.842970 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.842881 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f496b\" (UniqueName: \"kubernetes.io/projected/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-kube-api-access-f496b\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.842970 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.842933 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.842970 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.842963 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.843119 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.842988 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.843119 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.843103 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.843195 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.843149 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.944614 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.944579 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.944807 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.944648 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.944807 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.944690 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.944807 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.944731 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f496b\" (UniqueName: \"kubernetes.io/projected/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-kube-api-access-f496b\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.944807 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.944772 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.944807 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.944793 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.945101 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.944817 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.945569 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.945529 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.947416 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.947392 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.947416 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.947406 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.947587 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.947556 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.947587 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.947561 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.967960 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.967931 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.968092 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.967968 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz"] Jul 08 19:50:45.968092 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:45.967989 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f496b\" (UniqueName: \"kubernetes.io/projected/e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2-kube-api-access-f496b\") pod \"istiod-openshift-gateway-94bb8fbfd-g6bc2\" (UID: \"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:45.969787 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:50:45.969764 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod023f102d_9467_43d7_862d_57e07a82f63a.slice/crio-8368c6ae680927ff5078a5cfd73b8040aef7aa7fb105770a9248d7f01e0175b1 WatchSource:0}: Error finding container 8368c6ae680927ff5078a5cfd73b8040aef7aa7fb105770a9248d7f01e0175b1: Status 404 returned error can't find the container with id 8368c6ae680927ff5078a5cfd73b8040aef7aa7fb105770a9248d7f01e0175b1 Jul 08 19:50:46.076103 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:46.076072 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:46.216259 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:46.216224 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2"] Jul 08 19:50:46.219579 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:50:46.219548 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6709fe0_a57b_4f7f_8d1b_e2b764ff9ac2.slice/crio-dfb295a0a033e31b27b4095f9b7c67abdaca548d1bcad7e6b1fcbf8391fc0083 WatchSource:0}: Error finding container dfb295a0a033e31b27b4095f9b7c67abdaca548d1bcad7e6b1fcbf8391fc0083: Status 404 returned error can't find the container with id dfb295a0a033e31b27b4095f9b7c67abdaca548d1bcad7e6b1fcbf8391fc0083 Jul 08 19:50:46.906046 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:46.905992 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" event={"ID":"023f102d-9467-43d7-862d-57e07a82f63a","Type":"ContainerStarted","Data":"8368c6ae680927ff5078a5cfd73b8040aef7aa7fb105770a9248d7f01e0175b1"} Jul 08 19:50:46.907435 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:46.907410 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" event={"ID":"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2","Type":"ContainerStarted","Data":"dfb295a0a033e31b27b4095f9b7c67abdaca548d1bcad7e6b1fcbf8391fc0083"} Jul 08 19:50:48.830575 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:48.830538 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:49.111321 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:49.111217 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:49.261059 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:49.261011 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:49.544274 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:49.544186 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:50.098004 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:50.097956 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:50:50.098294 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:50.098050 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:50:50.512726 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:50.512635 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:50.800919 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:50.800836 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:50.901094 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:50.901057 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:50.931206 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:50.931177 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" event={"ID":"023f102d-9467-43d7-862d-57e07a82f63a","Type":"ContainerStarted","Data":"69643ec764c6a395662a478ffa7378ce03cab9af5b01161e49a3feb9bb89ee57"} Jul 08 19:50:50.931412 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:50.931289 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" Jul 08 19:50:50.932830 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:50.932808 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" event={"ID":"e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2","Type":"ContainerStarted","Data":"ddf752e79da0a01c066712486c783150ea7dfb68521bc1c49faaa954bcba0f9b"} Jul 08 19:50:50.932942 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:50.932871 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:50:50.954645 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:50.954588 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" podStartSLOduration=1.830586721 podStartE2EDuration="5.954572628s" totalImagesPullingTime="4.123985907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:50:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:50:50.952703765 +0000 UTC m=+532.646013251" watchObservedRunningTime="2026-07-08 19:50:50.954572628 +0000 UTC m=+532.647882114" Jul 08 19:50:50.975381 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:50.975333 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" podStartSLOduration=2.099455046 podStartE2EDuration="5.975317496s" totalImagesPullingTime="3.87586245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:50:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:50:50.973538943 +0000 UTC m=+532.666848437" watchObservedRunningTime="2026-07-08 19:50:50.975317496 +0000 UTC m=+532.668626976" Jul 08 19:50:51.422461 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:51.422424 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:51.651689 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:51.651651 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:51.729653 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:51.729565 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:51.939750 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:50:51.939724 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g6bc2" Jul 08 19:51:01.940322 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:51:01.940292 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7cffbd996-mjwfz" Jul 08 19:51:05.618425 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:51:05.618389 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:51:07.379162 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:51:07.379130 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:51:07.480960 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:51:07.480925 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:51:38.889683 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:51:38.889634 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:51:38.989395 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:51:38.989360 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:51:39.089105 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:51:39.089067 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:51:50.014367 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:51:50.014332 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:51:51.182882 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:51:51.182848 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:07.154736 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:07.154700 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:07.780969 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:07.780935 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:08.774897 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:08.774863 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:08.872417 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:08.872384 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:10.742181 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:10.742150 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:12.387994 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.387957 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:12.775469 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.775389 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79599f8676-4sd6g"] Jul 08 19:52:12.778745 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.778729 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.783728 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.783707 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-nqqwm\"" Jul 08 19:52:12.783863 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.783732 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 19:52:12.783863 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.783735 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 19:52:12.783863 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.783735 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 19:52:12.784174 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.784157 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 19:52:12.784261 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.784190 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 19:52:12.788753 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.788735 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 19:52:12.793120 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.793097 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79599f8676-4sd6g"] Jul 08 19:52:12.831153 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.831122 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:12.833112 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.833095 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzbsh\" (UniqueName: \"kubernetes.io/projected/86c56241-bacb-4608-abbc-cc356b88ec85-kube-api-access-jzbsh\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.833181 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.833123 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86c56241-bacb-4608-abbc-cc356b88ec85-console-config\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.833181 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.833139 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86c56241-bacb-4608-abbc-cc356b88ec85-trusted-ca-bundle\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.833181 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.833169 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86c56241-bacb-4608-abbc-cc356b88ec85-oauth-serving-cert\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.833340 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.833246 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86c56241-bacb-4608-abbc-cc356b88ec85-console-serving-cert\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.833340 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.833310 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86c56241-bacb-4608-abbc-cc356b88ec85-service-ca\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.833426 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.833360 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86c56241-bacb-4608-abbc-cc356b88ec85-console-oauth-config\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.934168 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.934130 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzbsh\" (UniqueName: \"kubernetes.io/projected/86c56241-bacb-4608-abbc-cc356b88ec85-kube-api-access-jzbsh\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.934358 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.934176 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86c56241-bacb-4608-abbc-cc356b88ec85-console-config\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.934358 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.934295 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86c56241-bacb-4608-abbc-cc356b88ec85-trusted-ca-bundle\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.934486 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.934368 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86c56241-bacb-4608-abbc-cc356b88ec85-oauth-serving-cert\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.934486 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.934403 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86c56241-bacb-4608-abbc-cc356b88ec85-console-serving-cert\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.934595 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.934485 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86c56241-bacb-4608-abbc-cc356b88ec85-service-ca\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.934595 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.934560 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86c56241-bacb-4608-abbc-cc356b88ec85-console-oauth-config\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.934996 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.934974 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86c56241-bacb-4608-abbc-cc356b88ec85-console-config\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.935153 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.935129 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86c56241-bacb-4608-abbc-cc356b88ec85-oauth-serving-cert\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.935286 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.935233 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86c56241-bacb-4608-abbc-cc356b88ec85-trusted-ca-bundle\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.935286 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.935253 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86c56241-bacb-4608-abbc-cc356b88ec85-service-ca\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.936876 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.936858 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86c56241-bacb-4608-abbc-cc356b88ec85-console-serving-cert\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.937092 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.937072 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86c56241-bacb-4608-abbc-cc356b88ec85-console-oauth-config\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:12.943568 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.943543 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:12.961988 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:12.961955 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzbsh\" (UniqueName: \"kubernetes.io/projected/86c56241-bacb-4608-abbc-cc356b88ec85-kube-api-access-jzbsh\") pod \"console-79599f8676-4sd6g\" (UID: \"86c56241-bacb-4608-abbc-cc356b88ec85\") " pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:13.064508 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:13.064471 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:13.088533 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:13.088497 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:13.234305 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:13.234280 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79599f8676-4sd6g"] Jul 08 19:52:13.236325 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:52:13.236297 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86c56241_bacb_4608_abbc_cc356b88ec85.slice/crio-76448298d2e1f2e00d620d8e0190b56c62d10edb44a849a12298820d3bc9b000 WatchSource:0}: Error finding container 76448298d2e1f2e00d620d8e0190b56c62d10edb44a849a12298820d3bc9b000: Status 404 returned error can't find the container with id 76448298d2e1f2e00d620d8e0190b56c62d10edb44a849a12298820d3bc9b000 Jul 08 19:52:13.323947 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:13.323859 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79599f8676-4sd6g" event={"ID":"86c56241-bacb-4608-abbc-cc356b88ec85","Type":"ContainerStarted","Data":"051ca1b151f3c469c627efe5bb357de2b5f85a6f9d06c22eb985dff7ddc7e2fa"} Jul 08 19:52:13.323947 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:13.323910 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79599f8676-4sd6g" event={"ID":"86c56241-bacb-4608-abbc-cc356b88ec85","Type":"ContainerStarted","Data":"76448298d2e1f2e00d620d8e0190b56c62d10edb44a849a12298820d3bc9b000"} Jul 08 19:52:13.349411 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:13.349350 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79599f8676-4sd6g" podStartSLOduration=1.349327542 podStartE2EDuration="1.349327542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:52:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:52:13.347274997 +0000 UTC m=+615.040584495" watchObservedRunningTime="2026-07-08 19:52:13.349327542 +0000 UTC m=+615.042637029" Jul 08 19:52:13.569030 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:13.568990 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:13.707185 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:13.707095 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:15.165602 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:15.165566 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:15.424365 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:15.424272 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:17.598987 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:17.598951 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:17.746835 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:17.746802 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:20.435290 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:20.435213 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:22.777459 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:22.777425 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:22.877339 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:22.877295 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:23.029703 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:23.029620 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:23.088776 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:23.088746 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:23.088957 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:23.088829 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:23.093507 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:23.093487 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:23.216535 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:23.216504 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:23.374792 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:23.374765 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79599f8676-4sd6g" Jul 08 19:52:23.577868 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:23.577838 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:23.676606 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:23.676517 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:23.906625 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:23.906593 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:26.609659 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:26.609626 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:26.707538 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:26.707504 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:30.594075 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:30.594019 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:30.680772 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:30.680735 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:30.893676 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:30.893585 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:31.142653 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:31.142622 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:31.243159 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:31.243068 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:34.186973 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:34.186942 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:34.300984 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:34.300950 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:40.498214 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:40.498177 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw"] Jul 08 19:52:40.501453 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:40.501436 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw" Jul 08 19:52:40.505863 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:40.505838 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gxwbh\"" Jul 08 19:52:40.505991 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:40.505867 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 19:52:40.506272 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:40.506257 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 19:52:40.518201 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:40.518174 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw"] Jul 08 19:52:40.584330 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:40.584297 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kwqk\" (UniqueName: \"kubernetes.io/projected/9f3e7db8-de44-4433-b2a3-51921c08151f-kube-api-access-7kwqk\") pod \"kuadrant-operator-controller-manager-55467d9f7-28kkw\" (UID: \"9f3e7db8-de44-4433-b2a3-51921c08151f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw" Jul 08 19:52:40.584492 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:40.584349 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f3e7db8-de44-4433-b2a3-51921c08151f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-28kkw\" (UID: \"9f3e7db8-de44-4433-b2a3-51921c08151f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw" Jul 08 19:52:40.685706 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:40.685659 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kwqk\" (UniqueName: \"kubernetes.io/projected/9f3e7db8-de44-4433-b2a3-51921c08151f-kube-api-access-7kwqk\") pod \"kuadrant-operator-controller-manager-55467d9f7-28kkw\" (UID: \"9f3e7db8-de44-4433-b2a3-51921c08151f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw" Jul 08 19:52:40.685845 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:40.685740 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f3e7db8-de44-4433-b2a3-51921c08151f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-28kkw\" (UID: \"9f3e7db8-de44-4433-b2a3-51921c08151f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw" Jul 08 19:52:40.686121 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:40.686105 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f3e7db8-de44-4433-b2a3-51921c08151f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-28kkw\" (UID: \"9f3e7db8-de44-4433-b2a3-51921c08151f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw" Jul 08 19:52:40.695393 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:40.695361 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kwqk\" (UniqueName: \"kubernetes.io/projected/9f3e7db8-de44-4433-b2a3-51921c08151f-kube-api-access-7kwqk\") pod \"kuadrant-operator-controller-manager-55467d9f7-28kkw\" (UID: \"9f3e7db8-de44-4433-b2a3-51921c08151f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw" Jul 08 19:52:40.812295 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:40.812262 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw" Jul 08 19:52:40.967674 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:40.967648 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw"] Jul 08 19:52:40.969908 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:52:40.969882 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f3e7db8_de44_4433_b2a3_51921c08151f.slice/crio-bacf50bdf6795e7e053294ea629235c0b22c8e06aa4f7aff43e82c0153fc892d WatchSource:0}: Error finding container bacf50bdf6795e7e053294ea629235c0b22c8e06aa4f7aff43e82c0153fc892d: Status 404 returned error can't find the container with id bacf50bdf6795e7e053294ea629235c0b22c8e06aa4f7aff43e82c0153fc892d Jul 08 19:52:40.972941 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:40.972923 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:52:41.045005 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:41.044968 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw"] Jul 08 19:52:41.055300 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:41.055276 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw"] Jul 08 19:52:41.073673 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:41.073594 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm"] Jul 08 19:52:41.083754 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:41.083726 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm" Jul 08 19:52:41.088377 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:41.088318 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftzhl\" (UniqueName: \"kubernetes.io/projected/01abc428-7ec0-4b56-b2b0-bfbe703657cf-kube-api-access-ftzhl\") pod \"kuadrant-operator-controller-manager-55467d9f7-hxplm\" (UID: \"01abc428-7ec0-4b56-b2b0-bfbe703657cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm" Jul 08 19:52:41.088544 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:41.088380 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/01abc428-7ec0-4b56-b2b0-bfbe703657cf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-hxplm\" (UID: \"01abc428-7ec0-4b56-b2b0-bfbe703657cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm" Jul 08 19:52:41.095496 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:41.095465 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm"] Jul 08 19:52:41.189522 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:41.189491 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftzhl\" (UniqueName: \"kubernetes.io/projected/01abc428-7ec0-4b56-b2b0-bfbe703657cf-kube-api-access-ftzhl\") pod \"kuadrant-operator-controller-manager-55467d9f7-hxplm\" (UID: \"01abc428-7ec0-4b56-b2b0-bfbe703657cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm" Jul 08 19:52:41.189696 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:41.189533 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/01abc428-7ec0-4b56-b2b0-bfbe703657cf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-hxplm\" (UID: \"01abc428-7ec0-4b56-b2b0-bfbe703657cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm" Jul 08 19:52:41.189914 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:41.189898 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/01abc428-7ec0-4b56-b2b0-bfbe703657cf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-hxplm\" (UID: \"01abc428-7ec0-4b56-b2b0-bfbe703657cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm" Jul 08 19:52:41.203372 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:41.203337 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftzhl\" (UniqueName: \"kubernetes.io/projected/01abc428-7ec0-4b56-b2b0-bfbe703657cf-kube-api-access-ftzhl\") pod \"kuadrant-operator-controller-manager-55467d9f7-hxplm\" (UID: \"01abc428-7ec0-4b56-b2b0-bfbe703657cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm" Jul 08 19:52:41.396802 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:41.396721 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm" Jul 08 19:52:41.552244 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:41.552077 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm"] Jul 08 19:52:41.557300 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:52:41.557265 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01abc428_7ec0_4b56_b2b0_bfbe703657cf.slice/crio-cc2cc4fc9f428a980f7a67853c104d73a0f61d68a059df30e2ae56adb4eef610 WatchSource:0}: Error finding container cc2cc4fc9f428a980f7a67853c104d73a0f61d68a059df30e2ae56adb4eef610: Status 404 returned error can't find the container with id cc2cc4fc9f428a980f7a67853c104d73a0f61d68a059df30e2ae56adb4eef610 Jul 08 19:52:42.394024 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:42.393983 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:42.469301 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:42.469264 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm" event={"ID":"01abc428-7ec0-4b56-b2b0-bfbe703657cf","Type":"ContainerStarted","Data":"cc2cc4fc9f428a980f7a67853c104d73a0f61d68a059df30e2ae56adb4eef610"} Jul 08 19:52:42.535534 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:42.535489 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:42.934642 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:42.934598 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:43.184260 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:43.184230 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:45.121782 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:45.121738 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:45.214576 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:45.214541 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:46.498488 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:46.498439 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw" podUID="9f3e7db8-de44-4433-b2a3-51921c08151f" containerName="manager" containerID="cri-o://49dcb46b7d5e4d7b68edf0f3940d2654a7a181ad3e0983d13e23788149e569ea" gracePeriod=2 Jul 08 19:52:46.500049 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:46.500005 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm" event={"ID":"01abc428-7ec0-4b56-b2b0-bfbe703657cf","Type":"ContainerStarted","Data":"67bc7e19c1a9ff5a9d299dd3240e5c4da1d754b35a99461f2b8454413603a5db"} Jul 08 19:52:46.500184 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:46.500123 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm" Jul 08 19:52:46.526805 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:46.526759 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm" podStartSLOduration=1.161581769 podStartE2EDuration="5.526743636s" totalImagesPullingTime="4.365161867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:52:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:52:46.524238218 +0000 UTC m=+648.217547707" watchObservedRunningTime="2026-07-08 19:52:46.526743636 +0000 UTC m=+648.220053121" Jul 08 19:52:46.757761 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:46.757696 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw" Jul 08 19:52:46.760431 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:52:46.760405 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-28kkw\" is forbidden: User \"system:node:ip-10-0-129-198.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-198.ec2.internal' and this object" podUID="9f3e7db8-de44-4433-b2a3-51921c08151f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw" Jul 08 19:52:46.844659 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:46.844627 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kwqk\" (UniqueName: \"kubernetes.io/projected/9f3e7db8-de44-4433-b2a3-51921c08151f-kube-api-access-7kwqk\") pod \"9f3e7db8-de44-4433-b2a3-51921c08151f\" (UID: \"9f3e7db8-de44-4433-b2a3-51921c08151f\") " Jul 08 19:52:46.844812 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:46.844753 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f3e7db8-de44-4433-b2a3-51921c08151f-extensions-socket-volume\") pod \"9f3e7db8-de44-4433-b2a3-51921c08151f\" (UID: \"9f3e7db8-de44-4433-b2a3-51921c08151f\") " Jul 08 19:52:46.845020 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:46.844994 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f3e7db8-de44-4433-b2a3-51921c08151f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9f3e7db8-de44-4433-b2a3-51921c08151f" (UID: "9f3e7db8-de44-4433-b2a3-51921c08151f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:52:46.846779 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:46.846758 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f3e7db8-de44-4433-b2a3-51921c08151f-kube-api-access-7kwqk" (OuterVolumeSpecName: "kube-api-access-7kwqk") pod "9f3e7db8-de44-4433-b2a3-51921c08151f" (UID: "9f3e7db8-de44-4433-b2a3-51921c08151f"). InnerVolumeSpecName "kube-api-access-7kwqk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:52:46.946105 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:46.946078 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kwqk\" (UniqueName: \"kubernetes.io/projected/9f3e7db8-de44-4433-b2a3-51921c08151f-kube-api-access-7kwqk\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:52:46.946105 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:46.946104 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f3e7db8-de44-4433-b2a3-51921c08151f-extensions-socket-volume\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:52:47.506337 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:47.506303 2568 generic.go:356] "Generic (PLEG): container finished" podID="9f3e7db8-de44-4433-b2a3-51921c08151f" containerID="49dcb46b7d5e4d7b68edf0f3940d2654a7a181ad3e0983d13e23788149e569ea" exitCode=2 Jul 08 19:52:47.506837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:47.506352 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw" Jul 08 19:52:47.506837 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:47.506397 2568 scope.go:117] "RemoveContainer" containerID="49dcb46b7d5e4d7b68edf0f3940d2654a7a181ad3e0983d13e23788149e569ea" Jul 08 19:52:47.516541 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:47.516526 2568 scope.go:117] "RemoveContainer" containerID="49dcb46b7d5e4d7b68edf0f3940d2654a7a181ad3e0983d13e23788149e569ea" Jul 08 19:52:47.516811 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:52:47.516793 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49dcb46b7d5e4d7b68edf0f3940d2654a7a181ad3e0983d13e23788149e569ea\": container with ID starting with 49dcb46b7d5e4d7b68edf0f3940d2654a7a181ad3e0983d13e23788149e569ea not found: ID does not exist" containerID="49dcb46b7d5e4d7b68edf0f3940d2654a7a181ad3e0983d13e23788149e569ea" Jul 08 19:52:47.516857 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:47.516821 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49dcb46b7d5e4d7b68edf0f3940d2654a7a181ad3e0983d13e23788149e569ea"} err="failed to get container status \"49dcb46b7d5e4d7b68edf0f3940d2654a7a181ad3e0983d13e23788149e569ea\": rpc error: code = NotFound desc = could not find container \"49dcb46b7d5e4d7b68edf0f3940d2654a7a181ad3e0983d13e23788149e569ea\": container with ID starting with 49dcb46b7d5e4d7b68edf0f3940d2654a7a181ad3e0983d13e23788149e569ea not found: ID does not exist" Jul 08 19:52:47.518225 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:52:47.518201 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-28kkw\" is forbidden: User \"system:node:ip-10-0-129-198.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-198.ec2.internal' and this object" podUID="9f3e7db8-de44-4433-b2a3-51921c08151f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw" Jul 08 19:52:48.831380 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:52:48.831342 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-28kkw\" is forbidden: User \"system:node:ip-10-0-129-198.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-198.ec2.internal' and this object" podUID="9f3e7db8-de44-4433-b2a3-51921c08151f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-28kkw" Jul 08 19:52:48.832116 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:48.832093 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f3e7db8-de44-4433-b2a3-51921c08151f" path="/var/lib/kubelet/pods/9f3e7db8-de44-4433-b2a3-51921c08151f/volumes" Jul 08 19:52:50.514818 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:50.514778 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:50.612450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:50.612421 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:53.925989 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:53.925951 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:54.029134 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:54.029103 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:52:57.508820 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:57.508790 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm" Jul 08 19:52:58.771152 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:52:58.771120 2568 scope.go:117] "RemoveContainer" containerID="f7050887514ecf7020698987aea49c14b0b5ec16bf7b600b4dffb6e3a39e2d25" Jul 08 19:53:03.474421 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:03.474385 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:03.547839 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:03.547807 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:05.858584 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:05.858549 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:09.402515 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:09.402479 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm"] Jul 08 19:53:09.402912 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:09.402687 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm" podUID="01abc428-7ec0-4b56-b2b0-bfbe703657cf" containerName="manager" containerID="cri-o://67bc7e19c1a9ff5a9d299dd3240e5c4da1d754b35a99461f2b8454413603a5db" gracePeriod=10 Jul 08 19:53:09.617781 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:09.617752 2568 generic.go:356] "Generic (PLEG): container finished" podID="01abc428-7ec0-4b56-b2b0-bfbe703657cf" containerID="67bc7e19c1a9ff5a9d299dd3240e5c4da1d754b35a99461f2b8454413603a5db" exitCode=0 Jul 08 19:53:09.617943 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:09.617787 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm" event={"ID":"01abc428-7ec0-4b56-b2b0-bfbe703657cf","Type":"ContainerDied","Data":"67bc7e19c1a9ff5a9d299dd3240e5c4da1d754b35a99461f2b8454413603a5db"} Jul 08 19:53:09.655840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:09.655777 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm" Jul 08 19:53:09.748542 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:09.748509 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/01abc428-7ec0-4b56-b2b0-bfbe703657cf-extensions-socket-volume\") pod \"01abc428-7ec0-4b56-b2b0-bfbe703657cf\" (UID: \"01abc428-7ec0-4b56-b2b0-bfbe703657cf\") " Jul 08 19:53:09.748709 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:09.748579 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftzhl\" (UniqueName: \"kubernetes.io/projected/01abc428-7ec0-4b56-b2b0-bfbe703657cf-kube-api-access-ftzhl\") pod \"01abc428-7ec0-4b56-b2b0-bfbe703657cf\" (UID: \"01abc428-7ec0-4b56-b2b0-bfbe703657cf\") " Jul 08 19:53:09.748980 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:09.748957 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01abc428-7ec0-4b56-b2b0-bfbe703657cf-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "01abc428-7ec0-4b56-b2b0-bfbe703657cf" (UID: "01abc428-7ec0-4b56-b2b0-bfbe703657cf"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:53:09.750726 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:09.750704 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01abc428-7ec0-4b56-b2b0-bfbe703657cf-kube-api-access-ftzhl" (OuterVolumeSpecName: "kube-api-access-ftzhl") pod "01abc428-7ec0-4b56-b2b0-bfbe703657cf" (UID: "01abc428-7ec0-4b56-b2b0-bfbe703657cf"). InnerVolumeSpecName "kube-api-access-ftzhl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:53:09.849356 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:09.849322 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/01abc428-7ec0-4b56-b2b0-bfbe703657cf-extensions-socket-volume\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:53:09.849356 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:09.849350 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftzhl\" (UniqueName: \"kubernetes.io/projected/01abc428-7ec0-4b56-b2b0-bfbe703657cf-kube-api-access-ftzhl\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:53:10.623768 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:10.623737 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm" Jul 08 19:53:10.624214 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:10.623737 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm" event={"ID":"01abc428-7ec0-4b56-b2b0-bfbe703657cf","Type":"ContainerDied","Data":"cc2cc4fc9f428a980f7a67853c104d73a0f61d68a059df30e2ae56adb4eef610"} Jul 08 19:53:10.624214 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:10.623869 2568 scope.go:117] "RemoveContainer" containerID="67bc7e19c1a9ff5a9d299dd3240e5c4da1d754b35a99461f2b8454413603a5db" Jul 08 19:53:10.653436 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:10.653407 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm"] Jul 08 19:53:10.662537 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:10.662514 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hxplm"] Jul 08 19:53:10.838290 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:10.838256 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01abc428-7ec0-4b56-b2b0-bfbe703657cf" path="/var/lib/kubelet/pods/01abc428-7ec0-4b56-b2b0-bfbe703657cf/volumes" Jul 08 19:53:14.081640 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:14.081603 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:14.313524 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:14.313491 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:31.939513 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:31.939480 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:31.940215 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:31.940195 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8sv2b"] Jul 08 19:53:31.940552 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:31.940540 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01abc428-7ec0-4b56-b2b0-bfbe703657cf" containerName="manager" Jul 08 19:53:31.940589 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:31.940554 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="01abc428-7ec0-4b56-b2b0-bfbe703657cf" containerName="manager" Jul 08 19:53:31.940589 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:31.940564 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f3e7db8-de44-4433-b2a3-51921c08151f" containerName="manager" Jul 08 19:53:31.940589 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:31.940569 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f3e7db8-de44-4433-b2a3-51921c08151f" containerName="manager" Jul 08 19:53:31.940687 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:31.940633 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="01abc428-7ec0-4b56-b2b0-bfbe703657cf" containerName="manager" Jul 08 19:53:31.940687 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:31.940645 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f3e7db8-de44-4433-b2a3-51921c08151f" containerName="manager" Jul 08 19:53:31.943667 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:31.943650 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8sv2b" Jul 08 19:53:31.947017 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:31.946999 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 19:53:31.947151 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:31.947058 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 08 19:53:31.948888 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:31.948871 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xl2pr\"" Jul 08 19:53:31.948988 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:31.948877 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 19:53:31.952548 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:31.952526 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8sv2b"] Jul 08 19:53:32.037464 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:32.037433 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8sv2b"] Jul 08 19:53:32.043002 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:32.042971 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:32.047968 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:32.047933 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/989fe5ea-7916-48ce-80f9-78dad44a04aa-config-file\") pod \"limitador-limitador-64c94fd64f-8sv2b\" (UID: \"989fe5ea-7916-48ce-80f9-78dad44a04aa\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8sv2b" Jul 08 19:53:32.048112 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:32.047997 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2ztt\" (UniqueName: \"kubernetes.io/projected/989fe5ea-7916-48ce-80f9-78dad44a04aa-kube-api-access-c2ztt\") pod \"limitador-limitador-64c94fd64f-8sv2b\" (UID: \"989fe5ea-7916-48ce-80f9-78dad44a04aa\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8sv2b" Jul 08 19:53:32.125789 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:32.125758 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:32.148804 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:32.148771 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/989fe5ea-7916-48ce-80f9-78dad44a04aa-config-file\") pod \"limitador-limitador-64c94fd64f-8sv2b\" (UID: \"989fe5ea-7916-48ce-80f9-78dad44a04aa\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8sv2b" Jul 08 19:53:32.148978 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:32.148822 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2ztt\" (UniqueName: \"kubernetes.io/projected/989fe5ea-7916-48ce-80f9-78dad44a04aa-kube-api-access-c2ztt\") pod \"limitador-limitador-64c94fd64f-8sv2b\" (UID: \"989fe5ea-7916-48ce-80f9-78dad44a04aa\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8sv2b" Jul 08 19:53:32.149510 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:32.149492 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/989fe5ea-7916-48ce-80f9-78dad44a04aa-config-file\") pod \"limitador-limitador-64c94fd64f-8sv2b\" (UID: \"989fe5ea-7916-48ce-80f9-78dad44a04aa\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8sv2b" Jul 08 19:53:32.160131 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:32.160098 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2ztt\" (UniqueName: \"kubernetes.io/projected/989fe5ea-7916-48ce-80f9-78dad44a04aa-kube-api-access-c2ztt\") pod \"limitador-limitador-64c94fd64f-8sv2b\" (UID: \"989fe5ea-7916-48ce-80f9-78dad44a04aa\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8sv2b" Jul 08 19:53:32.254645 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:32.254547 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8sv2b" Jul 08 19:53:32.393928 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:32.393903 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8sv2b"] Jul 08 19:53:32.395571 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:53:32.395545 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod989fe5ea_7916_48ce_80f9_78dad44a04aa.slice/crio-032e9f94709913a56c771d893affcb4041ccb2ce6e325982a7d0b81985976538 WatchSource:0}: Error finding container 032e9f94709913a56c771d893affcb4041ccb2ce6e325982a7d0b81985976538: Status 404 returned error can't find the container with id 032e9f94709913a56c771d893affcb4041ccb2ce6e325982a7d0b81985976538 Jul 08 19:53:32.737693 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:32.737662 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8sv2b" event={"ID":"989fe5ea-7916-48ce-80f9-78dad44a04aa","Type":"ContainerStarted","Data":"032e9f94709913a56c771d893affcb4041ccb2ce6e325982a7d0b81985976538"} Jul 08 19:53:32.888950 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:32.888865 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:32.958000 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:32.957966 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:35.756421 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:35.756384 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8sv2b" event={"ID":"989fe5ea-7916-48ce-80f9-78dad44a04aa","Type":"ContainerStarted","Data":"56c4ef13eb76bdee18ceda2b5e37128cd9fa7cc17231878ce29a87640b726d90"} Jul 08 19:53:35.756908 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:35.756522 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8sv2b" Jul 08 19:53:35.779930 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:35.779876 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-8sv2b" podStartSLOduration=2.270247564 podStartE2EDuration="4.779856594s" totalImagesPullingTime="2.50960903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:53:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:53:35.778239902 +0000 UTC m=+697.471549388" watchObservedRunningTime="2026-07-08 19:53:35.779856594 +0000 UTC m=+697.473166080" Jul 08 19:53:42.458516 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:42.458482 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:42.573964 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:42.573926 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:46.762583 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:46.762548 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8sv2b" Jul 08 19:53:48.690046 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:48.690010 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8sv2b"] Jul 08 19:53:48.690520 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:48.690218 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-8sv2b" podUID="989fe5ea-7916-48ce-80f9-78dad44a04aa" containerName="limitador" containerID="cri-o://56c4ef13eb76bdee18ceda2b5e37128cd9fa7cc17231878ce29a87640b726d90" gracePeriod=30 Jul 08 19:53:49.242154 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:49.242127 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8sv2b" Jul 08 19:53:49.312877 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:49.312849 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2ztt\" (UniqueName: \"kubernetes.io/projected/989fe5ea-7916-48ce-80f9-78dad44a04aa-kube-api-access-c2ztt\") pod \"989fe5ea-7916-48ce-80f9-78dad44a04aa\" (UID: \"989fe5ea-7916-48ce-80f9-78dad44a04aa\") " Jul 08 19:53:49.313082 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:49.312945 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/989fe5ea-7916-48ce-80f9-78dad44a04aa-config-file\") pod \"989fe5ea-7916-48ce-80f9-78dad44a04aa\" (UID: \"989fe5ea-7916-48ce-80f9-78dad44a04aa\") " Jul 08 19:53:49.313326 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:49.313302 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/989fe5ea-7916-48ce-80f9-78dad44a04aa-config-file" (OuterVolumeSpecName: "config-file") pod "989fe5ea-7916-48ce-80f9-78dad44a04aa" (UID: "989fe5ea-7916-48ce-80f9-78dad44a04aa"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:53:49.314979 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:49.314956 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/989fe5ea-7916-48ce-80f9-78dad44a04aa-kube-api-access-c2ztt" (OuterVolumeSpecName: "kube-api-access-c2ztt") pod "989fe5ea-7916-48ce-80f9-78dad44a04aa" (UID: "989fe5ea-7916-48ce-80f9-78dad44a04aa"). InnerVolumeSpecName "kube-api-access-c2ztt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:53:49.414710 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:49.414672 2568 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/989fe5ea-7916-48ce-80f9-78dad44a04aa-config-file\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:53:49.414710 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:49.414706 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2ztt\" (UniqueName: \"kubernetes.io/projected/989fe5ea-7916-48ce-80f9-78dad44a04aa-kube-api-access-c2ztt\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:53:49.825897 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:49.825857 2568 generic.go:356] "Generic (PLEG): container finished" podID="989fe5ea-7916-48ce-80f9-78dad44a04aa" containerID="56c4ef13eb76bdee18ceda2b5e37128cd9fa7cc17231878ce29a87640b726d90" exitCode=0 Jul 08 19:53:49.825897 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:49.825892 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8sv2b" Jul 08 19:53:49.826422 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:49.825946 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8sv2b" event={"ID":"989fe5ea-7916-48ce-80f9-78dad44a04aa","Type":"ContainerDied","Data":"56c4ef13eb76bdee18ceda2b5e37128cd9fa7cc17231878ce29a87640b726d90"} Jul 08 19:53:49.826422 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:49.825988 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8sv2b" event={"ID":"989fe5ea-7916-48ce-80f9-78dad44a04aa","Type":"ContainerDied","Data":"032e9f94709913a56c771d893affcb4041ccb2ce6e325982a7d0b81985976538"} Jul 08 19:53:49.826422 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:49.826010 2568 scope.go:117] "RemoveContainer" containerID="56c4ef13eb76bdee18ceda2b5e37128cd9fa7cc17231878ce29a87640b726d90" Jul 08 19:53:49.834949 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:49.834932 2568 scope.go:117] "RemoveContainer" containerID="56c4ef13eb76bdee18ceda2b5e37128cd9fa7cc17231878ce29a87640b726d90" Jul 08 19:53:49.835207 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:53:49.835187 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56c4ef13eb76bdee18ceda2b5e37128cd9fa7cc17231878ce29a87640b726d90\": container with ID starting with 56c4ef13eb76bdee18ceda2b5e37128cd9fa7cc17231878ce29a87640b726d90 not found: ID does not exist" containerID="56c4ef13eb76bdee18ceda2b5e37128cd9fa7cc17231878ce29a87640b726d90" Jul 08 19:53:49.835286 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:49.835218 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c4ef13eb76bdee18ceda2b5e37128cd9fa7cc17231878ce29a87640b726d90"} err="failed to get container status \"56c4ef13eb76bdee18ceda2b5e37128cd9fa7cc17231878ce29a87640b726d90\": rpc error: code = NotFound desc = could not find container \"56c4ef13eb76bdee18ceda2b5e37128cd9fa7cc17231878ce29a87640b726d90\": container with ID starting with 56c4ef13eb76bdee18ceda2b5e37128cd9fa7cc17231878ce29a87640b726d90 not found: ID does not exist" Jul 08 19:53:49.849169 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:49.849142 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8sv2b"] Jul 08 19:53:49.853698 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:49.853673 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8sv2b"] Jul 08 19:53:50.832157 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:50.832122 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="989fe5ea-7916-48ce-80f9-78dad44a04aa" path="/var/lib/kubelet/pods/989fe5ea-7916-48ce-80f9-78dad44a04aa/volumes" Jul 08 19:53:53.246951 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.246914 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:53.379158 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.379120 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:53.768758 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.768724 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fctlb"] Jul 08 19:53:53.769115 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.769102 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="989fe5ea-7916-48ce-80f9-78dad44a04aa" containerName="limitador" Jul 08 19:53:53.769115 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.769116 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="989fe5ea-7916-48ce-80f9-78dad44a04aa" containerName="limitador" Jul 08 19:53:53.769199 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.769193 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="989fe5ea-7916-48ce-80f9-78dad44a04aa" containerName="limitador" Jul 08 19:53:53.772239 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.772220 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fctlb" Jul 08 19:53:53.779127 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.779101 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 19:53:53.779244 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.779161 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 08 19:53:53.780561 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.780544 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 19:53:53.780628 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.780543 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-rs6zw\"" Jul 08 19:53:53.803280 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.803251 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fctlb"] Jul 08 19:53:53.809253 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.809222 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:53.852870 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.852837 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28x2n\" (UniqueName: \"kubernetes.io/projected/582feee9-e034-437f-8c20-8b50c668d621-kube-api-access-28x2n\") pod \"postgres-69fb4594cf-fctlb\" (UID: \"582feee9-e034-437f-8c20-8b50c668d621\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fctlb" Jul 08 19:53:53.853070 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.852877 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/582feee9-e034-437f-8c20-8b50c668d621-data\") pod \"postgres-69fb4594cf-fctlb\" (UID: \"582feee9-e034-437f-8c20-8b50c668d621\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fctlb" Jul 08 19:53:53.954006 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.953968 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28x2n\" (UniqueName: \"kubernetes.io/projected/582feee9-e034-437f-8c20-8b50c668d621-kube-api-access-28x2n\") pod \"postgres-69fb4594cf-fctlb\" (UID: \"582feee9-e034-437f-8c20-8b50c668d621\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fctlb" Jul 08 19:53:53.954006 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.954010 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/582feee9-e034-437f-8c20-8b50c668d621-data\") pod \"postgres-69fb4594cf-fctlb\" (UID: \"582feee9-e034-437f-8c20-8b50c668d621\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fctlb" Jul 08 19:53:53.954401 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.954383 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/582feee9-e034-437f-8c20-8b50c668d621-data\") pod \"postgres-69fb4594cf-fctlb\" (UID: \"582feee9-e034-437f-8c20-8b50c668d621\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fctlb" Jul 08 19:53:53.966536 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:53.966508 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28x2n\" (UniqueName: \"kubernetes.io/projected/582feee9-e034-437f-8c20-8b50c668d621-kube-api-access-28x2n\") pod \"postgres-69fb4594cf-fctlb\" (UID: \"582feee9-e034-437f-8c20-8b50c668d621\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fctlb" Jul 08 19:53:54.082671 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:54.082642 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fctlb" Jul 08 19:53:54.219083 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:54.219057 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fctlb"] Jul 08 19:53:54.220759 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:53:54.220728 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod582feee9_e034_437f_8c20_8b50c668d621.slice/crio-e41ba4b26a4d5875b5c9838890dd51ef1e37ee515b21d190a4d5a92c29bb3902 WatchSource:0}: Error finding container e41ba4b26a4d5875b5c9838890dd51ef1e37ee515b21d190a4d5a92c29bb3902: Status 404 returned error can't find the container with id e41ba4b26a4d5875b5c9838890dd51ef1e37ee515b21d190a4d5a92c29bb3902 Jul 08 19:53:54.852660 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:54.852607 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fctlb" event={"ID":"582feee9-e034-437f-8c20-8b50c668d621","Type":"ContainerStarted","Data":"e41ba4b26a4d5875b5c9838890dd51ef1e37ee515b21d190a4d5a92c29bb3902"} Jul 08 19:53:58.895681 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:58.895658 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 08 19:53:59.880485 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:59.880450 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fctlb" event={"ID":"582feee9-e034-437f-8c20-8b50c668d621","Type":"ContainerStarted","Data":"2bd91e42b216ecfc1b7c9c7d1e72af7710290df6195cb6a288c49c5d7265120d"} Jul 08 19:53:59.880681 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:59.880514 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fctlb" Jul 08 19:53:59.912903 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:53:59.912840 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fctlb" podStartSLOduration=2.244163777 podStartE2EDuration="6.912824315s" totalImagesPullingTime="4.668660538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:53:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:53:59.911348436 +0000 UTC m=+721.604657921" watchObservedRunningTime="2026-07-08 19:53:59.912824315 +0000 UTC m=+721.606133800" Jul 08 19:54:02.558177 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:02.558142 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:54:02.658244 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:02.658215 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:54:05.916378 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:05.916350 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fctlb" Jul 08 19:54:06.285871 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:06.285796 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:54:06.463819 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:06.463784 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:54:13.538720 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.538686 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:54:13.561823 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.561780 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-595cd57b6-hzx4n"] Jul 08 19:54:13.583716 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.583685 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-595cd57b6-hzx4n" Jul 08 19:54:13.588918 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.588895 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 08 19:54:13.588918 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.588914 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 08 19:54:13.589120 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.588952 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6lkr5\"" Jul 08 19:54:13.611679 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.611647 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-595cd57b6-hzx4n"] Jul 08 19:54:13.640647 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.640611 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2f5w\" (UniqueName: \"kubernetes.io/projected/497e084e-6e7d-4ea0-949f-0a78cb6cec8a-kube-api-access-n2f5w\") pod \"maas-controller-595cd57b6-hzx4n\" (UID: \"497e084e-6e7d-4ea0-949f-0a78cb6cec8a\") " pod="opendatahub/maas-controller-595cd57b6-hzx4n" Jul 08 19:54:13.640813 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.640680 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/497e084e-6e7d-4ea0-949f-0a78cb6cec8a-webhook-cert\") pod \"maas-controller-595cd57b6-hzx4n\" (UID: \"497e084e-6e7d-4ea0-949f-0a78cb6cec8a\") " pod="opendatahub/maas-controller-595cd57b6-hzx4n" Jul 08 19:54:13.741369 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.741336 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2f5w\" (UniqueName: \"kubernetes.io/projected/497e084e-6e7d-4ea0-949f-0a78cb6cec8a-kube-api-access-n2f5w\") pod \"maas-controller-595cd57b6-hzx4n\" (UID: \"497e084e-6e7d-4ea0-949f-0a78cb6cec8a\") " pod="opendatahub/maas-controller-595cd57b6-hzx4n" Jul 08 19:54:13.741552 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.741415 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/497e084e-6e7d-4ea0-949f-0a78cb6cec8a-webhook-cert\") pod \"maas-controller-595cd57b6-hzx4n\" (UID: \"497e084e-6e7d-4ea0-949f-0a78cb6cec8a\") " pod="opendatahub/maas-controller-595cd57b6-hzx4n" Jul 08 19:54:13.743759 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.743729 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/497e084e-6e7d-4ea0-949f-0a78cb6cec8a-webhook-cert\") pod \"maas-controller-595cd57b6-hzx4n\" (UID: \"497e084e-6e7d-4ea0-949f-0a78cb6cec8a\") " pod="opendatahub/maas-controller-595cd57b6-hzx4n" Jul 08 19:54:13.755553 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.755529 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2f5w\" (UniqueName: \"kubernetes.io/projected/497e084e-6e7d-4ea0-949f-0a78cb6cec8a-kube-api-access-n2f5w\") pod \"maas-controller-595cd57b6-hzx4n\" (UID: \"497e084e-6e7d-4ea0-949f-0a78cb6cec8a\") " pod="opendatahub/maas-controller-595cd57b6-hzx4n" Jul 08 19:54:13.766249 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.766226 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:54:13.879689 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.879659 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7984469857-7lkgf"] Jul 08 19:54:13.883257 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.883241 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7984469857-7lkgf" Jul 08 19:54:13.894468 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.894444 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-595cd57b6-hzx4n" Jul 08 19:54:13.908064 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.907990 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7984469857-7lkgf"] Jul 08 19:54:13.943209 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.943167 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b62b1888-640b-44d8-bad4-2e80dcd7024d-webhook-cert\") pod \"maas-controller-7984469857-7lkgf\" (UID: \"b62b1888-640b-44d8-bad4-2e80dcd7024d\") " pod="opendatahub/maas-controller-7984469857-7lkgf" Jul 08 19:54:13.943442 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.943300 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8fhh\" (UniqueName: \"kubernetes.io/projected/b62b1888-640b-44d8-bad4-2e80dcd7024d-kube-api-access-s8fhh\") pod \"maas-controller-7984469857-7lkgf\" (UID: \"b62b1888-640b-44d8-bad4-2e80dcd7024d\") " pod="opendatahub/maas-controller-7984469857-7lkgf" Jul 08 19:54:13.950014 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:13.949979 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:54:14.044625 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:14.044591 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8fhh\" (UniqueName: \"kubernetes.io/projected/b62b1888-640b-44d8-bad4-2e80dcd7024d-kube-api-access-s8fhh\") pod \"maas-controller-7984469857-7lkgf\" (UID: \"b62b1888-640b-44d8-bad4-2e80dcd7024d\") " pod="opendatahub/maas-controller-7984469857-7lkgf" Jul 08 19:54:14.044841 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:14.044695 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b62b1888-640b-44d8-bad4-2e80dcd7024d-webhook-cert\") pod \"maas-controller-7984469857-7lkgf\" (UID: \"b62b1888-640b-44d8-bad4-2e80dcd7024d\") " pod="opendatahub/maas-controller-7984469857-7lkgf" Jul 08 19:54:14.047604 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:14.047575 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b62b1888-640b-44d8-bad4-2e80dcd7024d-webhook-cert\") pod \"maas-controller-7984469857-7lkgf\" (UID: \"b62b1888-640b-44d8-bad4-2e80dcd7024d\") " pod="opendatahub/maas-controller-7984469857-7lkgf" Jul 08 19:54:14.049996 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:14.049978 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-595cd57b6-hzx4n"] Jul 08 19:54:14.051958 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:54:14.051932 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod497e084e_6e7d_4ea0_949f_0a78cb6cec8a.slice/crio-b87a9542017b16f29589bcc2a93103a2ef3c528587de9cee4e2568b31c171d34 WatchSource:0}: Error finding container b87a9542017b16f29589bcc2a93103a2ef3c528587de9cee4e2568b31c171d34: Status 404 returned error can't find the container with id b87a9542017b16f29589bcc2a93103a2ef3c528587de9cee4e2568b31c171d34 Jul 08 19:54:14.058105 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:14.058085 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8fhh\" (UniqueName: \"kubernetes.io/projected/b62b1888-640b-44d8-bad4-2e80dcd7024d-kube-api-access-s8fhh\") pod \"maas-controller-7984469857-7lkgf\" (UID: \"b62b1888-640b-44d8-bad4-2e80dcd7024d\") " pod="opendatahub/maas-controller-7984469857-7lkgf" Jul 08 19:54:14.194426 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:14.194330 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7984469857-7lkgf" Jul 08 19:54:14.345511 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:14.345487 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7984469857-7lkgf"] Jul 08 19:54:14.347522 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:54:14.347498 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb62b1888_640b_44d8_bad4_2e80dcd7024d.slice/crio-df4390abfd74666de838f9573593b2fc1adba67adf70f0e420e289f59bf9fe5d WatchSource:0}: Error finding container df4390abfd74666de838f9573593b2fc1adba67adf70f0e420e289f59bf9fe5d: Status 404 returned error can't find the container with id df4390abfd74666de838f9573593b2fc1adba67adf70f0e420e289f59bf9fe5d Jul 08 19:54:14.967506 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:14.967463 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-595cd57b6-hzx4n" event={"ID":"497e084e-6e7d-4ea0-949f-0a78cb6cec8a","Type":"ContainerStarted","Data":"b87a9542017b16f29589bcc2a93103a2ef3c528587de9cee4e2568b31c171d34"} Jul 08 19:54:14.969361 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:14.969331 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7984469857-7lkgf" event={"ID":"b62b1888-640b-44d8-bad4-2e80dcd7024d","Type":"ContainerStarted","Data":"df4390abfd74666de838f9573593b2fc1adba67adf70f0e420e289f59bf9fe5d"} Jul 08 19:54:16.582741 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:16.582672 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:54:16.798671 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:16.798637 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:54:16.981658 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:16.981561 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7984469857-7lkgf" event={"ID":"b62b1888-640b-44d8-bad4-2e80dcd7024d","Type":"ContainerStarted","Data":"917ad6a2cb714292feb63c9639f93681955a2831f7efed1c8d5c7c917d6ef88f"} Jul 08 19:54:16.981658 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:16.981651 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7984469857-7lkgf" Jul 08 19:54:16.983045 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:16.983011 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-595cd57b6-hzx4n" event={"ID":"497e084e-6e7d-4ea0-949f-0a78cb6cec8a","Type":"ContainerStarted","Data":"10112249992cdcffdf61b8894fc9aea241f17438def79b5d1c3fd9cb30b95508"} Jul 08 19:54:16.983150 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:16.983135 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-595cd57b6-hzx4n" Jul 08 19:54:17.046962 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:17.046908 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7984469857-7lkgf" podStartSLOduration=2.159072054 podStartE2EDuration="4.046891571s" totalImagesPullingTime="1.887819517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:54:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:54:17.044278693 +0000 UTC m=+738.737588179" watchObservedRunningTime="2026-07-08 19:54:17.046891571 +0000 UTC m=+738.740201056" Jul 08 19:54:17.084018 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:17.083965 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-595cd57b6-hzx4n" podStartSLOduration=1.890141802 podStartE2EDuration="4.083948419s" totalImagesPullingTime="2.193806617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:54:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:54:17.081415805 +0000 UTC m=+738.774725302" watchObservedRunningTime="2026-07-08 19:54:17.083948419 +0000 UTC m=+738.777257905" Jul 08 19:54:27.995872 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:27.995831 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-595cd57b6-hzx4n" Jul 08 19:54:27.996336 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:27.996064 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7984469857-7lkgf" Jul 08 19:54:28.098554 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:28.098525 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-595cd57b6-hzx4n"] Jul 08 19:54:28.098777 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:28.098753 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-595cd57b6-hzx4n" podUID="497e084e-6e7d-4ea0-949f-0a78cb6cec8a" containerName="manager" containerID="cri-o://10112249992cdcffdf61b8894fc9aea241f17438def79b5d1c3fd9cb30b95508" gracePeriod=30 Jul 08 19:54:28.349251 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:28.349226 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-595cd57b6-hzx4n" Jul 08 19:54:28.486522 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:28.486487 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/497e084e-6e7d-4ea0-949f-0a78cb6cec8a-webhook-cert\") pod \"497e084e-6e7d-4ea0-949f-0a78cb6cec8a\" (UID: \"497e084e-6e7d-4ea0-949f-0a78cb6cec8a\") " Jul 08 19:54:28.486694 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:28.486587 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2f5w\" (UniqueName: \"kubernetes.io/projected/497e084e-6e7d-4ea0-949f-0a78cb6cec8a-kube-api-access-n2f5w\") pod \"497e084e-6e7d-4ea0-949f-0a78cb6cec8a\" (UID: \"497e084e-6e7d-4ea0-949f-0a78cb6cec8a\") " Jul 08 19:54:28.488800 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:28.488762 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/497e084e-6e7d-4ea0-949f-0a78cb6cec8a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "497e084e-6e7d-4ea0-949f-0a78cb6cec8a" (UID: "497e084e-6e7d-4ea0-949f-0a78cb6cec8a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:54:28.488919 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:28.488815 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/497e084e-6e7d-4ea0-949f-0a78cb6cec8a-kube-api-access-n2f5w" (OuterVolumeSpecName: "kube-api-access-n2f5w") pod "497e084e-6e7d-4ea0-949f-0a78cb6cec8a" (UID: "497e084e-6e7d-4ea0-949f-0a78cb6cec8a"). InnerVolumeSpecName "kube-api-access-n2f5w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:54:28.587467 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:28.587435 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2f5w\" (UniqueName: \"kubernetes.io/projected/497e084e-6e7d-4ea0-949f-0a78cb6cec8a-kube-api-access-n2f5w\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:54:28.587467 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:28.587469 2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/497e084e-6e7d-4ea0-949f-0a78cb6cec8a-webhook-cert\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:54:29.042870 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:29.042837 2568 generic.go:356] "Generic (PLEG): container finished" podID="497e084e-6e7d-4ea0-949f-0a78cb6cec8a" containerID="10112249992cdcffdf61b8894fc9aea241f17438def79b5d1c3fd9cb30b95508" exitCode=0 Jul 08 19:54:29.043285 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:29.042882 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-595cd57b6-hzx4n" Jul 08 19:54:29.043285 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:29.042922 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-595cd57b6-hzx4n" event={"ID":"497e084e-6e7d-4ea0-949f-0a78cb6cec8a","Type":"ContainerDied","Data":"10112249992cdcffdf61b8894fc9aea241f17438def79b5d1c3fd9cb30b95508"} Jul 08 19:54:29.043285 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:29.042977 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-595cd57b6-hzx4n" event={"ID":"497e084e-6e7d-4ea0-949f-0a78cb6cec8a","Type":"ContainerDied","Data":"b87a9542017b16f29589bcc2a93103a2ef3c528587de9cee4e2568b31c171d34"} Jul 08 19:54:29.043285 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:29.043006 2568 scope.go:117] "RemoveContainer" containerID="10112249992cdcffdf61b8894fc9aea241f17438def79b5d1c3fd9cb30b95508" Jul 08 19:54:29.051744 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:29.051724 2568 scope.go:117] "RemoveContainer" containerID="10112249992cdcffdf61b8894fc9aea241f17438def79b5d1c3fd9cb30b95508" Jul 08 19:54:29.052061 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:54:29.052015 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10112249992cdcffdf61b8894fc9aea241f17438def79b5d1c3fd9cb30b95508\": container with ID starting with 10112249992cdcffdf61b8894fc9aea241f17438def79b5d1c3fd9cb30b95508 not found: ID does not exist" containerID="10112249992cdcffdf61b8894fc9aea241f17438def79b5d1c3fd9cb30b95508" Jul 08 19:54:29.052130 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:29.052070 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10112249992cdcffdf61b8894fc9aea241f17438def79b5d1c3fd9cb30b95508"} err="failed to get container status \"10112249992cdcffdf61b8894fc9aea241f17438def79b5d1c3fd9cb30b95508\": rpc error: code = NotFound desc = could not find container \"10112249992cdcffdf61b8894fc9aea241f17438def79b5d1c3fd9cb30b95508\": container with ID starting with 10112249992cdcffdf61b8894fc9aea241f17438def79b5d1c3fd9cb30b95508 not found: ID does not exist" Jul 08 19:54:29.066019 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:29.065987 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-595cd57b6-hzx4n"] Jul 08 19:54:29.071421 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:29.071392 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-595cd57b6-hzx4n"] Jul 08 19:54:30.832222 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:30.832192 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="497e084e-6e7d-4ea0-949f-0a78cb6cec8a" path="/var/lib/kubelet/pods/497e084e-6e7d-4ea0-949f-0a78cb6cec8a/volumes" Jul 08 19:54:49.291380 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:49.291311 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:54:49.816025 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:49.815981 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:54:49.862772 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:49.862731 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6"] Jul 08 19:54:49.863220 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:49.863202 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="497e084e-6e7d-4ea0-949f-0a78cb6cec8a" containerName="manager" Jul 08 19:54:49.863326 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:49.863222 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="497e084e-6e7d-4ea0-949f-0a78cb6cec8a" containerName="manager" Jul 08 19:54:49.863387 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:49.863329 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="497e084e-6e7d-4ea0-949f-0a78cb6cec8a" containerName="manager" Jul 08 19:54:49.867860 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:49.867838 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6" Jul 08 19:54:49.874714 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:49.874690 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 08 19:54:49.874838 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:49.874744 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-sb6sh\"" Jul 08 19:54:49.888691 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:49.888657 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93673bbe-72c5-4150-b9fc-3af1caf633b2-openshift-service-ca\") pod \"maas-api-7dc9d84f6b-vmbz6\" (UID: \"93673bbe-72c5-4150-b9fc-3af1caf633b2\") " pod="odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6" Jul 08 19:54:49.888864 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:49.888718 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93673bbe-72c5-4150-b9fc-3af1caf633b2-maas-api-tls\") pod \"maas-api-7dc9d84f6b-vmbz6\" (UID: \"93673bbe-72c5-4150-b9fc-3af1caf633b2\") " pod="odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6" Jul 08 19:54:49.888864 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:49.888760 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpglq\" (UniqueName: \"kubernetes.io/projected/93673bbe-72c5-4150-b9fc-3af1caf633b2-kube-api-access-tpglq\") pod \"maas-api-7dc9d84f6b-vmbz6\" (UID: \"93673bbe-72c5-4150-b9fc-3af1caf633b2\") " pod="odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6" Jul 08 19:54:49.942974 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:49.942941 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6"] Jul 08 19:54:49.990319 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:49.990278 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93673bbe-72c5-4150-b9fc-3af1caf633b2-openshift-service-ca\") pod \"maas-api-7dc9d84f6b-vmbz6\" (UID: \"93673bbe-72c5-4150-b9fc-3af1caf633b2\") " pod="odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6" Jul 08 19:54:49.990528 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:49.990334 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93673bbe-72c5-4150-b9fc-3af1caf633b2-maas-api-tls\") pod \"maas-api-7dc9d84f6b-vmbz6\" (UID: \"93673bbe-72c5-4150-b9fc-3af1caf633b2\") " pod="odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6" Jul 08 19:54:49.990528 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:49.990375 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpglq\" (UniqueName: \"kubernetes.io/projected/93673bbe-72c5-4150-b9fc-3af1caf633b2-kube-api-access-tpglq\") pod \"maas-api-7dc9d84f6b-vmbz6\" (UID: \"93673bbe-72c5-4150-b9fc-3af1caf633b2\") " pod="odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6" Jul 08 19:54:49.990528 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:54:49.990503 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 08 19:54:49.990684 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:54:49.990589 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93673bbe-72c5-4150-b9fc-3af1caf633b2-maas-api-tls podName:93673bbe-72c5-4150-b9fc-3af1caf633b2 nodeName:}" failed. No retries permitted until 2026-07-08 19:54:50.490568815 +0000 UTC m=+772.183878280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/93673bbe-72c5-4150-b9fc-3af1caf633b2-maas-api-tls") pod "maas-api-7dc9d84f6b-vmbz6" (UID: "93673bbe-72c5-4150-b9fc-3af1caf633b2") : secret "maas-api-serving-cert" not found Jul 08 19:54:49.990971 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:49.990953 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93673bbe-72c5-4150-b9fc-3af1caf633b2-openshift-service-ca\") pod \"maas-api-7dc9d84f6b-vmbz6\" (UID: \"93673bbe-72c5-4150-b9fc-3af1caf633b2\") " pod="odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6" Jul 08 19:54:50.016298 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:50.016268 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpglq\" (UniqueName: \"kubernetes.io/projected/93673bbe-72c5-4150-b9fc-3af1caf633b2-kube-api-access-tpglq\") pod \"maas-api-7dc9d84f6b-vmbz6\" (UID: \"93673bbe-72c5-4150-b9fc-3af1caf633b2\") " pod="odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6" Jul 08 19:54:50.291807 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:50.291721 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:54:50.493950 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:50.493916 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93673bbe-72c5-4150-b9fc-3af1caf633b2-maas-api-tls\") pod \"maas-api-7dc9d84f6b-vmbz6\" (UID: \"93673bbe-72c5-4150-b9fc-3af1caf633b2\") " pod="odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6" Jul 08 19:54:50.496307 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:50.496287 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93673bbe-72c5-4150-b9fc-3af1caf633b2-maas-api-tls\") pod \"maas-api-7dc9d84f6b-vmbz6\" (UID: \"93673bbe-72c5-4150-b9fc-3af1caf633b2\") " pod="odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6" Jul 08 19:54:50.778223 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:50.778189 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6" Jul 08 19:54:51.015881 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.015854 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6"] Jul 08 19:54:51.017087 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:54:51.017044 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93673bbe_72c5_4150_b9fc_3af1caf633b2.slice/crio-53b6fe2c0a897d4258f615fe5fe19399b9d19e7c8c951c75221582187d285074 WatchSource:0}: Error finding container 53b6fe2c0a897d4258f615fe5fe19399b9d19e7c8c951c75221582187d285074: Status 404 returned error can't find the container with id 53b6fe2c0a897d4258f615fe5fe19399b9d19e7c8c951c75221582187d285074 Jul 08 19:54:51.042711 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.042641 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:54:51.156840 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.156807 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6" event={"ID":"93673bbe-72c5-4150-b9fc-3af1caf633b2","Type":"ContainerStarted","Data":"53b6fe2c0a897d4258f615fe5fe19399b9d19e7c8c951c75221582187d285074"} Jul 08 19:54:51.232818 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.232781 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:54:51.288611 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.288573 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-g5476"] Jul 08 19:54:51.294253 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.294201 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-g5476" Jul 08 19:54:51.300617 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.300593 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lftrc\" (UniqueName: \"kubernetes.io/projected/9ec1fb2b-9530-49e6-98d6-3002e9e0c589-kube-api-access-lftrc\") pod \"payload-pre-processing-7964457696-g5476\" (UID: \"9ec1fb2b-9530-49e6-98d6-3002e9e0c589\") " pod="openshift-ingress/payload-pre-processing-7964457696-g5476" Jul 08 19:54:51.300735 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.300679 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ec1fb2b-9530-49e6-98d6-3002e9e0c589-plugins-config-volume\") pod \"payload-pre-processing-7964457696-g5476\" (UID: \"9ec1fb2b-9530-49e6-98d6-3002e9e0c589\") " pod="openshift-ingress/payload-pre-processing-7964457696-g5476" Jul 08 19:54:51.324905 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.324876 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-74j5t\"" Jul 08 19:54:51.324905 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.324908 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 08 19:54:51.326923 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.326895 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-g5476"] Jul 08 19:54:51.401396 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.401367 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ec1fb2b-9530-49e6-98d6-3002e9e0c589-plugins-config-volume\") pod \"payload-pre-processing-7964457696-g5476\" (UID: \"9ec1fb2b-9530-49e6-98d6-3002e9e0c589\") " pod="openshift-ingress/payload-pre-processing-7964457696-g5476" Jul 08 19:54:51.401572 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.401420 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lftrc\" (UniqueName: \"kubernetes.io/projected/9ec1fb2b-9530-49e6-98d6-3002e9e0c589-kube-api-access-lftrc\") pod \"payload-pre-processing-7964457696-g5476\" (UID: \"9ec1fb2b-9530-49e6-98d6-3002e9e0c589\") " pod="openshift-ingress/payload-pre-processing-7964457696-g5476" Jul 08 19:54:51.402158 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.402140 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ec1fb2b-9530-49e6-98d6-3002e9e0c589-plugins-config-volume\") pod \"payload-pre-processing-7964457696-g5476\" (UID: \"9ec1fb2b-9530-49e6-98d6-3002e9e0c589\") " pod="openshift-ingress/payload-pre-processing-7964457696-g5476" Jul 08 19:54:51.422950 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.422914 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lftrc\" (UniqueName: \"kubernetes.io/projected/9ec1fb2b-9530-49e6-98d6-3002e9e0c589-kube-api-access-lftrc\") pod \"payload-pre-processing-7964457696-g5476\" (UID: \"9ec1fb2b-9530-49e6-98d6-3002e9e0c589\") " pod="openshift-ingress/payload-pre-processing-7964457696-g5476" Jul 08 19:54:51.499525 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.499491 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-7rw4p"] Jul 08 19:54:51.503611 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.503584 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-7rw4p" Jul 08 19:54:51.533002 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.532963 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-7rw4p"] Jul 08 19:54:51.603122 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.603085 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx7lw\" (UniqueName: \"kubernetes.io/projected/2350b9d0-b68f-44cd-9bed-954ba64b7685-kube-api-access-mx7lw\") pod \"payload-processing-58f86c7798-7rw4p\" (UID: \"2350b9d0-b68f-44cd-9bed-954ba64b7685\") " pod="openshift-ingress/payload-processing-58f86c7798-7rw4p" Jul 08 19:54:51.603297 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.603256 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2350b9d0-b68f-44cd-9bed-954ba64b7685-plugins-config-volume\") pod \"payload-processing-58f86c7798-7rw4p\" (UID: \"2350b9d0-b68f-44cd-9bed-954ba64b7685\") " pod="openshift-ingress/payload-processing-58f86c7798-7rw4p" Jul 08 19:54:51.604881 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.604861 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-g5476" Jul 08 19:54:51.704752 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.704721 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx7lw\" (UniqueName: \"kubernetes.io/projected/2350b9d0-b68f-44cd-9bed-954ba64b7685-kube-api-access-mx7lw\") pod \"payload-processing-58f86c7798-7rw4p\" (UID: \"2350b9d0-b68f-44cd-9bed-954ba64b7685\") " pod="openshift-ingress/payload-processing-58f86c7798-7rw4p" Jul 08 19:54:51.704917 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.704839 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2350b9d0-b68f-44cd-9bed-954ba64b7685-plugins-config-volume\") pod \"payload-processing-58f86c7798-7rw4p\" (UID: \"2350b9d0-b68f-44cd-9bed-954ba64b7685\") " pod="openshift-ingress/payload-processing-58f86c7798-7rw4p" Jul 08 19:54:51.705639 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.705616 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2350b9d0-b68f-44cd-9bed-954ba64b7685-plugins-config-volume\") pod \"payload-processing-58f86c7798-7rw4p\" (UID: \"2350b9d0-b68f-44cd-9bed-954ba64b7685\") " pod="openshift-ingress/payload-processing-58f86c7798-7rw4p" Jul 08 19:54:51.720799 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.720770 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx7lw\" (UniqueName: \"kubernetes.io/projected/2350b9d0-b68f-44cd-9bed-954ba64b7685-kube-api-access-mx7lw\") pod \"payload-processing-58f86c7798-7rw4p\" (UID: \"2350b9d0-b68f-44cd-9bed-954ba64b7685\") " pod="openshift-ingress/payload-processing-58f86c7798-7rw4p" Jul 08 19:54:51.814455 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.814411 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-7rw4p" Jul 08 19:54:51.817219 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.817191 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-g5476"] Jul 08 19:54:51.820581 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:54:51.820550 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ec1fb2b_9530_49e6_98d6_3002e9e0c589.slice/crio-1100fa0936c23beb8417944f21392cd2e61512e7dcd9127991f5eb39e95e35e5 WatchSource:0}: Error finding container 1100fa0936c23beb8417944f21392cd2e61512e7dcd9127991f5eb39e95e35e5: Status 404 returned error can't find the container with id 1100fa0936c23beb8417944f21392cd2e61512e7dcd9127991f5eb39e95e35e5 Jul 08 19:54:51.979145 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:51.979116 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-7rw4p"] Jul 08 19:54:51.981403 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:54:51.981372 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2350b9d0_b68f_44cd_9bed_954ba64b7685.slice/crio-c1bcafe24a41fc6a9227b374fa78dab14c01386157ed8787f3aa8729a5d85e2a WatchSource:0}: Error finding container c1bcafe24a41fc6a9227b374fa78dab14c01386157ed8787f3aa8729a5d85e2a: Status 404 returned error can't find the container with id c1bcafe24a41fc6a9227b374fa78dab14c01386157ed8787f3aa8729a5d85e2a Jul 08 19:54:52.162199 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:52.162105 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-g5476" event={"ID":"9ec1fb2b-9530-49e6-98d6-3002e9e0c589","Type":"ContainerStarted","Data":"1100fa0936c23beb8417944f21392cd2e61512e7dcd9127991f5eb39e95e35e5"} Jul 08 19:54:52.163241 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:52.163219 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-7rw4p" event={"ID":"2350b9d0-b68f-44cd-9bed-954ba64b7685","Type":"ContainerStarted","Data":"c1bcafe24a41fc6a9227b374fa78dab14c01386157ed8787f3aa8729a5d85e2a"} Jul 08 19:54:53.171295 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:53.171256 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6" event={"ID":"93673bbe-72c5-4150-b9fc-3af1caf633b2","Type":"ContainerStarted","Data":"c6d92838a629e84890553ccb3f0570c442584a20a73f2dad1a1c72594940c7e1"} Jul 08 19:54:53.171759 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:53.171346 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6" Jul 08 19:54:54.177973 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:54.177939 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-g5476" event={"ID":"9ec1fb2b-9530-49e6-98d6-3002e9e0c589","Type":"ContainerStarted","Data":"48fb5d7b31dc98855702e53fac9c4cc1f41a33951f09887fbfc7826e7019457c"} Jul 08 19:54:54.178450 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:54.177997 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-g5476" Jul 08 19:54:54.179564 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:54.179540 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-7rw4p" event={"ID":"2350b9d0-b68f-44cd-9bed-954ba64b7685","Type":"ContainerStarted","Data":"9b5b8d8c0f93632db974dcab2cedffb53256e953ffb3e23b5c6f14c0e9cd5c1c"} Jul 08 19:54:54.179725 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:54.179707 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-7rw4p" Jul 08 19:54:54.210705 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:54.210658 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6" podStartSLOduration=3.516859181 podStartE2EDuration="5.21064174s" totalImagesPullingTime="1.693782559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:54:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:54:53.218197212 +0000 UTC m=+774.911506699" watchObservedRunningTime="2026-07-08 19:54:54.21064174 +0000 UTC m=+775.903951233" Jul 08 19:54:54.211737 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:54.211705 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-g5476" podStartSLOduration=1.675733187 podStartE2EDuration="3.21169326s" totalImagesPullingTime="1.535960073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:54:54.209715487 +0000 UTC m=+775.903024972" watchObservedRunningTime="2026-07-08 19:54:54.21169326 +0000 UTC m=+775.905002746" Jul 08 19:54:54.242958 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:54.242907 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-7rw4p" podStartSLOduration=1.863715429 podStartE2EDuration="3.242891072s" totalImagesPullingTime="1.379175643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:54:54.242378191 +0000 UTC m=+775.935687677" watchObservedRunningTime="2026-07-08 19:54:54.242891072 +0000 UTC m=+775.936200558" Jul 08 19:54:59.185456 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:54:59.185384 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7dc9d84f6b-vmbz6" Jul 08 19:55:00.266332 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:00.266296 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:55:00.402355 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:00.402323 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:55:05.186061 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:05.186003 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-7rw4p" Jul 08 19:55:05.186568 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:05.186351 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-g5476" Jul 08 19:55:07.445086 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.445018 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck"] Jul 08 19:55:07.453937 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.453912 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.457319 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.457293 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sqmkz\"" Jul 08 19:55:07.457464 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.457400 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 19:55:07.458851 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.458837 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 08 19:55:07.458922 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.458838 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 19:55:07.481308 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.481274 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck"] Jul 08 19:55:07.556577 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.556543 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:55:07.567238 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.567213 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4faffbf-2c56-4073-a404-6468008a73b8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck\" (UID: \"c4faffbf-2c56-4073-a404-6468008a73b8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.567369 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.567248 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4faffbf-2c56-4073-a404-6468008a73b8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck\" (UID: \"c4faffbf-2c56-4073-a404-6468008a73b8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.567369 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.567342 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4faffbf-2c56-4073-a404-6468008a73b8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck\" (UID: \"c4faffbf-2c56-4073-a404-6468008a73b8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.567485 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.567398 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2n6h\" (UniqueName: \"kubernetes.io/projected/c4faffbf-2c56-4073-a404-6468008a73b8-kube-api-access-p2n6h\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck\" (UID: \"c4faffbf-2c56-4073-a404-6468008a73b8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.567485 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.567466 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4faffbf-2c56-4073-a404-6468008a73b8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck\" (UID: \"c4faffbf-2c56-4073-a404-6468008a73b8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.567568 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.567524 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4faffbf-2c56-4073-a404-6468008a73b8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck\" (UID: \"c4faffbf-2c56-4073-a404-6468008a73b8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.668732 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.668694 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4faffbf-2c56-4073-a404-6468008a73b8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck\" (UID: \"c4faffbf-2c56-4073-a404-6468008a73b8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.668909 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.668771 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4faffbf-2c56-4073-a404-6468008a73b8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck\" (UID: \"c4faffbf-2c56-4073-a404-6468008a73b8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.668909 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.668800 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4faffbf-2c56-4073-a404-6468008a73b8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck\" (UID: \"c4faffbf-2c56-4073-a404-6468008a73b8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.668909 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.668860 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4faffbf-2c56-4073-a404-6468008a73b8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck\" (UID: \"c4faffbf-2c56-4073-a404-6468008a73b8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.668909 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.668891 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2n6h\" (UniqueName: \"kubernetes.io/projected/c4faffbf-2c56-4073-a404-6468008a73b8-kube-api-access-p2n6h\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck\" (UID: \"c4faffbf-2c56-4073-a404-6468008a73b8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.669157 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.668917 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4faffbf-2c56-4073-a404-6468008a73b8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck\" (UID: \"c4faffbf-2c56-4073-a404-6468008a73b8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.669211 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.669160 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4faffbf-2c56-4073-a404-6468008a73b8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck\" (UID: \"c4faffbf-2c56-4073-a404-6468008a73b8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.669211 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.669189 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4faffbf-2c56-4073-a404-6468008a73b8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck\" (UID: \"c4faffbf-2c56-4073-a404-6468008a73b8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.669300 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.669241 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4faffbf-2c56-4073-a404-6468008a73b8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck\" (UID: \"c4faffbf-2c56-4073-a404-6468008a73b8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.671251 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.671224 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4faffbf-2c56-4073-a404-6468008a73b8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck\" (UID: \"c4faffbf-2c56-4073-a404-6468008a73b8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.671383 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.671365 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4faffbf-2c56-4073-a404-6468008a73b8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck\" (UID: \"c4faffbf-2c56-4073-a404-6468008a73b8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.680595 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.680574 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2n6h\" (UniqueName: \"kubernetes.io/projected/c4faffbf-2c56-4073-a404-6468008a73b8-kube-api-access-p2n6h\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck\" (UID: \"c4faffbf-2c56-4073-a404-6468008a73b8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.764801 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.764691 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:07.910013 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:07.909986 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck"] Jul 08 19:55:07.910829 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:55:07.910800 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4faffbf_2c56_4073_a404_6468008a73b8.slice/crio-698db0807ade71ccc987ca1c3e4483d46171dce2e2fd176ecec032721f32fb4f WatchSource:0}: Error finding container 698db0807ade71ccc987ca1c3e4483d46171dce2e2fd176ecec032721f32fb4f: Status 404 returned error can't find the container with id 698db0807ade71ccc987ca1c3e4483d46171dce2e2fd176ecec032721f32fb4f Jul 08 19:55:08.250510 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:08.250455 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" event={"ID":"c4faffbf-2c56-4073-a404-6468008a73b8","Type":"ContainerStarted","Data":"698db0807ade71ccc987ca1c3e4483d46171dce2e2fd176ecec032721f32fb4f"} Jul 08 19:55:09.256804 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:09.256765 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" event={"ID":"c4faffbf-2c56-4073-a404-6468008a73b8","Type":"ContainerStarted","Data":"48d9b0e17655be706a43d4a7358f4b919569a46f65ea08101cc35f31f3650e88"} Jul 08 19:55:09.290279 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:09.290226 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" podStartSLOduration=1.1462432630000001 podStartE2EDuration="2.290210877s" totalImagesPullingTime="1.143967614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:55:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:55:09.289012924 +0000 UTC m=+790.982322412" watchObservedRunningTime="2026-07-08 19:55:09.290210877 +0000 UTC m=+790.983520364" Jul 08 19:55:17.461893 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:17.461820 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:55:17.765588 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:17.765493 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:17.765745 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:17.765584 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:17.778159 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:17.778132 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:18.319384 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:18.319355 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck" Jul 08 19:55:32.806115 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:32.806019 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b"] Jul 08 19:55:32.811639 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:32.811616 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:32.815375 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:32.815342 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 08 19:55:32.825510 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:32.825479 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b"] Jul 08 19:55:32.853338 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:32.853304 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:55:32.913617 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:32.913581 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b496dbb8-9e0b-4947-9b2f-464dd1b4c47b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgp7b\" (UID: \"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:32.913617 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:32.913627 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b496dbb8-9e0b-4947-9b2f-464dd1b4c47b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgp7b\" (UID: \"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:32.913901 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:32.913694 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b496dbb8-9e0b-4947-9b2f-464dd1b4c47b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgp7b\" (UID: \"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:32.913901 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:32.913786 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b496dbb8-9e0b-4947-9b2f-464dd1b4c47b-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgp7b\" (UID: \"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:32.913901 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:32.913853 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qpbq\" (UniqueName: \"kubernetes.io/projected/b496dbb8-9e0b-4947-9b2f-464dd1b4c47b-kube-api-access-8qpbq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgp7b\" (UID: \"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:32.913901 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:32.913886 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b496dbb8-9e0b-4947-9b2f-464dd1b4c47b-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgp7b\" (UID: \"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:33.014957 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:33.014919 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qpbq\" (UniqueName: \"kubernetes.io/projected/b496dbb8-9e0b-4947-9b2f-464dd1b4c47b-kube-api-access-8qpbq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgp7b\" (UID: \"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:33.014957 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:33.014959 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b496dbb8-9e0b-4947-9b2f-464dd1b4c47b-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgp7b\" (UID: \"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:33.015221 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:33.014981 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b496dbb8-9e0b-4947-9b2f-464dd1b4c47b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgp7b\" (UID: \"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:33.015221 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:33.015000 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b496dbb8-9e0b-4947-9b2f-464dd1b4c47b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgp7b\" (UID: \"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:33.015221 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:33.015126 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b496dbb8-9e0b-4947-9b2f-464dd1b4c47b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgp7b\" (UID: \"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:33.015376 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:33.015230 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b496dbb8-9e0b-4947-9b2f-464dd1b4c47b-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgp7b\" (UID: \"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:33.015429 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:33.015403 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b496dbb8-9e0b-4947-9b2f-464dd1b4c47b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgp7b\" (UID: \"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:33.015484 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:33.015461 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b496dbb8-9e0b-4947-9b2f-464dd1b4c47b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgp7b\" (UID: \"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:33.015576 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:33.015554 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b496dbb8-9e0b-4947-9b2f-464dd1b4c47b-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgp7b\" (UID: \"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:33.017330 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:33.017307 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b496dbb8-9e0b-4947-9b2f-464dd1b4c47b-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgp7b\" (UID: \"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:33.017726 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:33.017704 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b496dbb8-9e0b-4947-9b2f-464dd1b4c47b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgp7b\" (UID: \"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:33.026030 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:33.026008 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qpbq\" (UniqueName: \"kubernetes.io/projected/b496dbb8-9e0b-4947-9b2f-464dd1b4c47b-kube-api-access-8qpbq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgp7b\" (UID: \"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:33.124783 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:33.124741 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:33.263111 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:33.263079 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b"] Jul 08 19:55:33.264999 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:55:33.264971 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb496dbb8_9e0b_4947_9b2f_464dd1b4c47b.slice/crio-a5c21d2c96e8db70e3de085845487bdef20f9f4d0da7c7a157fe785d357112f3 WatchSource:0}: Error finding container a5c21d2c96e8db70e3de085845487bdef20f9f4d0da7c7a157fe785d357112f3: Status 404 returned error can't find the container with id a5c21d2c96e8db70e3de085845487bdef20f9f4d0da7c7a157fe785d357112f3 Jul 08 19:55:33.378999 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:33.378909 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" event={"ID":"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b","Type":"ContainerStarted","Data":"a5c21d2c96e8db70e3de085845487bdef20f9f4d0da7c7a157fe785d357112f3"} Jul 08 19:55:34.385789 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:34.385749 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" event={"ID":"b496dbb8-9e0b-4947-9b2f-464dd1b4c47b","Type":"ContainerStarted","Data":"38ee87198de92f7155596664c23bf2cde7649dd21fc035c2e5ebcd4bed5a354f"} Jul 08 19:55:34.406586 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:34.406535 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" podStartSLOduration=2.169143711 podStartE2EDuration="2.406517986s" totalImagesPullingTime="237.374275ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:55:34.404998505 +0000 UTC m=+816.098307992" watchObservedRunningTime="2026-07-08 19:55:34.406517986 +0000 UTC m=+816.099827471" Jul 08 19:55:40.396118 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.396081 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl"] Jul 08 19:55:40.399822 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.399804 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.402721 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.402698 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 08 19:55:40.407995 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.407975 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl"] Jul 08 19:55:40.486579 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.486544 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88813fd9-c794-4b90-842e-eaf3dc357196-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl\" (UID: \"88813fd9-c794-4b90-842e-eaf3dc357196\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.486745 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.486591 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88813fd9-c794-4b90-842e-eaf3dc357196-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl\" (UID: \"88813fd9-c794-4b90-842e-eaf3dc357196\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.486745 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.486617 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqwhl\" (UniqueName: \"kubernetes.io/projected/88813fd9-c794-4b90-842e-eaf3dc357196-kube-api-access-hqwhl\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl\" (UID: \"88813fd9-c794-4b90-842e-eaf3dc357196\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.486745 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.486673 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88813fd9-c794-4b90-842e-eaf3dc357196-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl\" (UID: \"88813fd9-c794-4b90-842e-eaf3dc357196\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.486745 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.486711 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88813fd9-c794-4b90-842e-eaf3dc357196-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl\" (UID: \"88813fd9-c794-4b90-842e-eaf3dc357196\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.486885 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.486811 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88813fd9-c794-4b90-842e-eaf3dc357196-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl\" (UID: \"88813fd9-c794-4b90-842e-eaf3dc357196\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.500391 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.500362 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:55:40.588066 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.588017 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88813fd9-c794-4b90-842e-eaf3dc357196-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl\" (UID: \"88813fd9-c794-4b90-842e-eaf3dc357196\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.588255 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.588114 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88813fd9-c794-4b90-842e-eaf3dc357196-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl\" (UID: \"88813fd9-c794-4b90-842e-eaf3dc357196\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.588255 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.588145 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88813fd9-c794-4b90-842e-eaf3dc357196-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl\" (UID: \"88813fd9-c794-4b90-842e-eaf3dc357196\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.588382 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.588342 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqwhl\" (UniqueName: \"kubernetes.io/projected/88813fd9-c794-4b90-842e-eaf3dc357196-kube-api-access-hqwhl\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl\" (UID: \"88813fd9-c794-4b90-842e-eaf3dc357196\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.588446 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.588397 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88813fd9-c794-4b90-842e-eaf3dc357196-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl\" (UID: \"88813fd9-c794-4b90-842e-eaf3dc357196\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.588446 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.588433 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88813fd9-c794-4b90-842e-eaf3dc357196-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl\" (UID: \"88813fd9-c794-4b90-842e-eaf3dc357196\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.588556 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.588439 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88813fd9-c794-4b90-842e-eaf3dc357196-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl\" (UID: \"88813fd9-c794-4b90-842e-eaf3dc357196\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.588556 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.588512 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88813fd9-c794-4b90-842e-eaf3dc357196-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl\" (UID: \"88813fd9-c794-4b90-842e-eaf3dc357196\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.588768 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.588749 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88813fd9-c794-4b90-842e-eaf3dc357196-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl\" (UID: \"88813fd9-c794-4b90-842e-eaf3dc357196\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.590659 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.590636 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88813fd9-c794-4b90-842e-eaf3dc357196-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl\" (UID: \"88813fd9-c794-4b90-842e-eaf3dc357196\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.591158 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.591138 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88813fd9-c794-4b90-842e-eaf3dc357196-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl\" (UID: \"88813fd9-c794-4b90-842e-eaf3dc357196\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.597156 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.597132 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqwhl\" (UniqueName: \"kubernetes.io/projected/88813fd9-c794-4b90-842e-eaf3dc357196-kube-api-access-hqwhl\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl\" (UID: \"88813fd9-c794-4b90-842e-eaf3dc357196\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.712202 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.712119 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:40.852705 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:40.852675 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl"] Jul 08 19:55:40.854693 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:55:40.854670 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88813fd9_c794_4b90_842e_eaf3dc357196.slice/crio-615081f8ba31e9bb2819243c297b28d033fd423805514cd141f57caef7987c64 WatchSource:0}: Error finding container 615081f8ba31e9bb2819243c297b28d033fd423805514cd141f57caef7987c64: Status 404 returned error can't find the container with id 615081f8ba31e9bb2819243c297b28d033fd423805514cd141f57caef7987c64 Jul 08 19:55:41.423689 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:41.423653 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" event={"ID":"88813fd9-c794-4b90-842e-eaf3dc357196","Type":"ContainerStarted","Data":"348ae79198d9ebd932a3193d180fd606e959135eb2fb8eae34df0f664664d803"} Jul 08 19:55:41.424102 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:41.423707 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" event={"ID":"88813fd9-c794-4b90-842e-eaf3dc357196","Type":"ContainerStarted","Data":"615081f8ba31e9bb2819243c297b28d033fd423805514cd141f57caef7987c64"} Jul 08 19:55:41.443382 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:41.443336 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" podStartSLOduration=1.263283081 podStartE2EDuration="1.443318078s" totalImagesPullingTime="180.034997ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:55:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:55:41.442465965 +0000 UTC m=+823.135775451" watchObservedRunningTime="2026-07-08 19:55:41.443318078 +0000 UTC m=+823.136627568" Jul 08 19:55:43.125649 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:43.125609 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:43.126175 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:43.125696 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:43.137988 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:43.137964 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:43.445665 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:43.445582 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgp7b" Jul 08 19:55:45.835337 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:45.835296 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:55:50.713166 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:50.713127 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:50.713576 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:50.713198 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:50.725623 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:50.725596 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:51.487048 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:51.487010 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl" Jul 08 19:55:56.597014 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:55:56.596982 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:56:18.684139 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:56:18.684100 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:56:18.785518 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:56:18.785483 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:56:35.844224 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:56:35.844186 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:56:35.942320 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:56:35.942284 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:57:11.414458 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:57:11.414422 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:57:12.984360 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:57:12.984323 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:58:08.464638 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:08.464600 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:58:17.422930 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:17.422892 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:58:24.065533 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:24.065452 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:58:24.169114 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:24.169080 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7984469857-7lkgf"] Jul 08 19:58:24.169343 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:24.169318 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7984469857-7lkgf" podUID="b62b1888-640b-44d8-bad4-2e80dcd7024d" containerName="manager" containerID="cri-o://917ad6a2cb714292feb63c9639f93681955a2831f7efed1c8d5c7c917d6ef88f" gracePeriod=30 Jul 08 19:58:24.418818 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:24.418790 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7984469857-7lkgf" Jul 08 19:58:24.509347 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:24.509312 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b62b1888-640b-44d8-bad4-2e80dcd7024d-webhook-cert\") pod \"b62b1888-640b-44d8-bad4-2e80dcd7024d\" (UID: \"b62b1888-640b-44d8-bad4-2e80dcd7024d\") " Jul 08 19:58:24.509534 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:24.509366 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8fhh\" (UniqueName: \"kubernetes.io/projected/b62b1888-640b-44d8-bad4-2e80dcd7024d-kube-api-access-s8fhh\") pod \"b62b1888-640b-44d8-bad4-2e80dcd7024d\" (UID: \"b62b1888-640b-44d8-bad4-2e80dcd7024d\") " Jul 08 19:58:24.511558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:24.511523 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b62b1888-640b-44d8-bad4-2e80dcd7024d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b62b1888-640b-44d8-bad4-2e80dcd7024d" (UID: "b62b1888-640b-44d8-bad4-2e80dcd7024d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:58:24.511558 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:24.511542 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b62b1888-640b-44d8-bad4-2e80dcd7024d-kube-api-access-s8fhh" (OuterVolumeSpecName: "kube-api-access-s8fhh") pod "b62b1888-640b-44d8-bad4-2e80dcd7024d" (UID: "b62b1888-640b-44d8-bad4-2e80dcd7024d"). InnerVolumeSpecName "kube-api-access-s8fhh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:58:24.610534 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:24.610443 2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b62b1888-640b-44d8-bad4-2e80dcd7024d-webhook-cert\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:58:24.610534 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:24.610474 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8fhh\" (UniqueName: \"kubernetes.io/projected/b62b1888-640b-44d8-bad4-2e80dcd7024d-kube-api-access-s8fhh\") on node \"ip-10-0-129-198.ec2.internal\" DevicePath \"\"" Jul 08 19:58:25.256720 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:25.256683 2568 generic.go:356] "Generic (PLEG): container finished" podID="b62b1888-640b-44d8-bad4-2e80dcd7024d" containerID="917ad6a2cb714292feb63c9639f93681955a2831f7efed1c8d5c7c917d6ef88f" exitCode=0 Jul 08 19:58:25.257190 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:25.256741 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7984469857-7lkgf" Jul 08 19:58:25.257190 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:25.256771 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7984469857-7lkgf" event={"ID":"b62b1888-640b-44d8-bad4-2e80dcd7024d","Type":"ContainerDied","Data":"917ad6a2cb714292feb63c9639f93681955a2831f7efed1c8d5c7c917d6ef88f"} Jul 08 19:58:25.257190 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:25.256851 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7984469857-7lkgf" event={"ID":"b62b1888-640b-44d8-bad4-2e80dcd7024d","Type":"ContainerDied","Data":"df4390abfd74666de838f9573593b2fc1adba67adf70f0e420e289f59bf9fe5d"} Jul 08 19:58:25.257190 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:25.256874 2568 scope.go:117] "RemoveContainer" containerID="917ad6a2cb714292feb63c9639f93681955a2831f7efed1c8d5c7c917d6ef88f" Jul 08 19:58:25.265763 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:25.265746 2568 scope.go:117] "RemoveContainer" containerID="917ad6a2cb714292feb63c9639f93681955a2831f7efed1c8d5c7c917d6ef88f" Jul 08 19:58:25.266075 ip-10-0-129-198 kubenswrapper[2568]: E0708 19:58:25.266056 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"917ad6a2cb714292feb63c9639f93681955a2831f7efed1c8d5c7c917d6ef88f\": container with ID starting with 917ad6a2cb714292feb63c9639f93681955a2831f7efed1c8d5c7c917d6ef88f not found: ID does not exist" containerID="917ad6a2cb714292feb63c9639f93681955a2831f7efed1c8d5c7c917d6ef88f" Jul 08 19:58:25.266131 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:25.266083 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"917ad6a2cb714292feb63c9639f93681955a2831f7efed1c8d5c7c917d6ef88f"} err="failed to get container status \"917ad6a2cb714292feb63c9639f93681955a2831f7efed1c8d5c7c917d6ef88f\": rpc error: code = NotFound desc = could not find container \"917ad6a2cb714292feb63c9639f93681955a2831f7efed1c8d5c7c917d6ef88f\": container with ID starting with 917ad6a2cb714292feb63c9639f93681955a2831f7efed1c8d5c7c917d6ef88f not found: ID does not exist" Jul 08 19:58:25.277666 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:25.277636 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7984469857-7lkgf"] Jul 08 19:58:25.281934 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:25.281908 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7984469857-7lkgf"] Jul 08 19:58:25.956912 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:25.956885 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:58:26.082175 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.082140 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7984469857-gdrg2"] Jul 08 19:58:26.082564 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.082551 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b62b1888-640b-44d8-bad4-2e80dcd7024d" containerName="manager" Jul 08 19:58:26.082611 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.082565 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b62b1888-640b-44d8-bad4-2e80dcd7024d" containerName="manager" Jul 08 19:58:26.082660 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.082650 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b62b1888-640b-44d8-bad4-2e80dcd7024d" containerName="manager" Jul 08 19:58:26.086970 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.086955 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7984469857-gdrg2" Jul 08 19:58:26.090713 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.090692 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6lkr5\"" Jul 08 19:58:26.091370 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.091350 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 08 19:58:26.092136 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.092117 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 08 19:58:26.102285 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.102262 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7984469857-gdrg2"] Jul 08 19:58:26.226318 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.226228 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx84s\" (UniqueName: \"kubernetes.io/projected/8a7afc52-7de8-4884-81b3-0cdab99f23fa-kube-api-access-sx84s\") pod \"maas-controller-7984469857-gdrg2\" (UID: \"8a7afc52-7de8-4884-81b3-0cdab99f23fa\") " pod="opendatahub/maas-controller-7984469857-gdrg2" Jul 08 19:58:26.226318 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.226297 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a7afc52-7de8-4884-81b3-0cdab99f23fa-webhook-cert\") pod \"maas-controller-7984469857-gdrg2\" (UID: \"8a7afc52-7de8-4884-81b3-0cdab99f23fa\") " pod="opendatahub/maas-controller-7984469857-gdrg2" Jul 08 19:58:26.327315 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.327278 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx84s\" (UniqueName: \"kubernetes.io/projected/8a7afc52-7de8-4884-81b3-0cdab99f23fa-kube-api-access-sx84s\") pod \"maas-controller-7984469857-gdrg2\" (UID: \"8a7afc52-7de8-4884-81b3-0cdab99f23fa\") " pod="opendatahub/maas-controller-7984469857-gdrg2" Jul 08 19:58:26.327688 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.327358 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a7afc52-7de8-4884-81b3-0cdab99f23fa-webhook-cert\") pod \"maas-controller-7984469857-gdrg2\" (UID: \"8a7afc52-7de8-4884-81b3-0cdab99f23fa\") " pod="opendatahub/maas-controller-7984469857-gdrg2" Jul 08 19:58:26.329822 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.329803 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a7afc52-7de8-4884-81b3-0cdab99f23fa-webhook-cert\") pod \"maas-controller-7984469857-gdrg2\" (UID: \"8a7afc52-7de8-4884-81b3-0cdab99f23fa\") " pod="opendatahub/maas-controller-7984469857-gdrg2" Jul 08 19:58:26.337263 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.337241 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx84s\" (UniqueName: \"kubernetes.io/projected/8a7afc52-7de8-4884-81b3-0cdab99f23fa-kube-api-access-sx84s\") pod \"maas-controller-7984469857-gdrg2\" (UID: \"8a7afc52-7de8-4884-81b3-0cdab99f23fa\") " pod="opendatahub/maas-controller-7984469857-gdrg2" Jul 08 19:58:26.397369 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.397333 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7984469857-gdrg2" Jul 08 19:58:26.535771 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.535748 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7984469857-gdrg2"] Jul 08 19:58:26.537917 ip-10-0-129-198 kubenswrapper[2568]: W0708 19:58:26.537889 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a7afc52_7de8_4884_81b3_0cdab99f23fa.slice/crio-443b4d600022cbb00bccf2cadbbeeb4cd2d438f0bfd24f987e1866fe02f3ce6f WatchSource:0}: Error finding container 443b4d600022cbb00bccf2cadbbeeb4cd2d438f0bfd24f987e1866fe02f3ce6f: Status 404 returned error can't find the container with id 443b4d600022cbb00bccf2cadbbeeb4cd2d438f0bfd24f987e1866fe02f3ce6f Jul 08 19:58:26.539343 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.539321 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:58:26.833480 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:26.833448 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b62b1888-640b-44d8-bad4-2e80dcd7024d" path="/var/lib/kubelet/pods/b62b1888-640b-44d8-bad4-2e80dcd7024d/volumes" Jul 08 19:58:27.269684 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:27.269597 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7984469857-gdrg2" event={"ID":"8a7afc52-7de8-4884-81b3-0cdab99f23fa","Type":"ContainerStarted","Data":"5e8c5e00183d75c5e3a6b0784eb3e194a2f70990c441784a0000c208f65c0fe0"} Jul 08 19:58:27.269684 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:27.269646 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7984469857-gdrg2" event={"ID":"8a7afc52-7de8-4884-81b3-0cdab99f23fa","Type":"ContainerStarted","Data":"443b4d600022cbb00bccf2cadbbeeb4cd2d438f0bfd24f987e1866fe02f3ce6f"} Jul 08 19:58:27.269875 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:27.269705 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7984469857-gdrg2" Jul 08 19:58:38.279827 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:38.279790 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7984469857-gdrg2" Jul 08 19:58:38.299081 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:58:38.299009 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7984469857-gdrg2" podStartSLOduration=11.903414313 podStartE2EDuration="12.298992918s" totalImagesPullingTime="395.578605ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:58:27.293055403 +0000 UTC m=+988.986364896" watchObservedRunningTime="2026-07-08 19:58:38.298992918 +0000 UTC m=+999.992302405" Jul 08 19:59:04.821921 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:59:04.821881 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:59:04.923112 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:59:04.923075 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:59:05.033083 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:59:05.033025 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:59:10.874701 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:59:10.874659 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:59:10.948405 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:59:10.948374 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:59:11.046049 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:59:11.046004 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:59:11.152359 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:59:11.152270 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:59:11.247471 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:59:11.247432 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:59:17.285047 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:59:17.284999 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:59:17.386714 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:59:17.386674 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 19:59:34.238559 ip-10-0-129-198 kubenswrapper[2568]: I0708 19:59:34.238526 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:02:42.041708 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:02:42.041675 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:04:02.830229 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:04:02.830193 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:04:38.429600 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:04:38.429565 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:05:14.099312 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:05:14.099279 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:09:07.369779 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:09:07.369739 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:04.323802 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:04.323762 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:04.422705 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:04.422672 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:05.835277 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:05.835225 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:13.254791 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:13.254749 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:21.517198 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:21.517113 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:26.652149 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:26.652108 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:29.491588 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:29.491548 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:29.579447 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:29.579417 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:29.995972 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:29.995940 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:35.836355 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:35.836318 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:35.913634 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:35.913601 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:36.133878 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:36.133785 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:36.237454 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:36.237427 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:37.250873 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:37.250822 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:37.350718 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:37.350684 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:37.548918 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:37.548828 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:37.699780 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:37.699750 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:38.022721 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:38.022690 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:38.135233 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:38.135201 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:40.603668 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:40.603636 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:42.420953 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:42.420894 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:42.611223 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:42.611190 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:42.708809 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:42.708725 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:42.906895 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:42.906859 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:44.486083 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:44.486044 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:48.708843 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:48.708811 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:48.818448 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:48.818411 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:48.969100 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:48.968997 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:49.065113 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:49.065085 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:50.171858 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:50.171819 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:50.308050 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:50.307994 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:50.472684 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:50.472603 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:50.687054 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:50.687012 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:51.074023 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:51.073989 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:51.173157 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:51.173124 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:52.357179 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:52.357145 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:55.177701 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:55.177659 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:10:55.560304 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:10:55.560218 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:01.314733 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:01.314695 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:01.489942 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:01.489911 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:03.764012 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:03.763953 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:07.278619 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:07.278578 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:07.357222 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:07.357186 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:07.591777 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:07.591735 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:07.725825 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:07.725791 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:08.721662 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:08.721609 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:08.820021 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:08.819985 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:08.908805 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:08.908774 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:09.168640 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:09.168607 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:09.284606 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:09.284572 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:09.390017 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:09.389985 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:10.937870 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:10.937838 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:13.101279 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:13.101247 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:13.338304 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:13.338271 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:13.469155 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:13.469056 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:13.628978 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:13.628942 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:17.117945 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:17.117912 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:19.547710 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:19.547675 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:19.693659 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:19.693630 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:20.018093 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:20.018059 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:20.112747 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:20.112711 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:20.213664 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:20.213630 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:21.576204 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:21.576165 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:27.777739 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:27.777703 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:27.883171 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:27.883144 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:27.979284 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:27.979248 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:28.085601 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:28.085571 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:28.471811 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:28.471723 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:11:54.431198 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:11:54.431163 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:00.289381 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:00.289339 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:00.394692 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:00.394653 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:00.528674 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:00.528635 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:16.402703 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:16.402668 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:22.505632 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:22.505589 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:22.607519 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:22.607438 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:22.705422 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:22.705377 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:33.252472 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:33.252429 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:34.042028 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:34.041989 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:39.705318 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:39.705245 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:39.815327 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:39.815273 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:39.913820 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:39.913769 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:40.005868 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:40.005748 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:40.845806 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:40.845759 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:41.000678 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:41.000647 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:42.561875 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:42.561837 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:48.401346 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:48.401289 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:48.480278 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:48.480229 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:49.097088 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:49.097018 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:55.068401 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:55.068354 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:55.125535 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:55.125494 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:55.301599 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:55.301566 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:55.624141 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:55.624100 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:56.012844 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:56.012753 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:58.821967 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:58.821927 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:58.917557 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:58.917521 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:59.003338 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:59.003298 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:59.175242 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:59.175152 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:59.375253 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:59.375223 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:12:59.479546 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:12:59.479458 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:03.305216 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:03.305178 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:03.957397 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:03.957358 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:04.051443 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:04.051413 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:04.375231 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:04.375201 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:10.095878 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:10.095836 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:10.155127 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:10.155098 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:10.514459 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:10.514366 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:10.854347 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:10.854316 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:10.955050 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:10.955000 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:16.815977 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:16.815938 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:16.913957 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:16.913925 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:17.084988 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:17.084906 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:17.425340 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:17.425256 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:17.525310 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:17.525278 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:17.648475 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:17.648442 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:23.517840 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:23.517752 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:23.618718 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:23.618683 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:23.718566 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:23.718523 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:23.841931 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:23.841902 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:29.585282 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:29.585246 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:29.690523 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:29.690496 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:36.349249 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:36.349202 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:36.722537 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:36.722453 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:37.225132 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:37.225103 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:39.406526 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:39.406495 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:40.063155 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:40.063123 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:40.160553 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:40.160516 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:40.322872 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:40.322792 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:40.421927 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:40.421893 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:40.864630 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:40.864596 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:40.961988 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:40.961953 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:44.579634 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:44.579599 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:46.642733 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:46.642705 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:13:46.742293 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:13:46.742261 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:14:01.707217 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:14:01.707175 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:14:01.805360 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:14:01.805317 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:14:02.295503 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:14:02.295468 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:14:22.091435 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:14:22.091398 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:14:23.462377 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:14:23.462342 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:14:28.146400 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:14:28.146345 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:14:29.378811 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:14:29.378778 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:14:33.181700 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:14:33.181655 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:14:41.815544 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:14:41.815510 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:09.702994 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:09.702955 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:10.646267 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:10.646234 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:16.402864 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:16.402823 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:39.085301 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:39.085263 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:39.185506 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:39.185473 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:39.322442 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:39.322412 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:40.600017 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:40.599981 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:40.698906 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:40.698875 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:40.799142 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:40.799107 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:41.495283 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:41.495251 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:41.589717 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:41.589678 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:41.689351 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:41.689319 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:43.047464 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:43.047430 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:48.795079 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:48.795018 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:48.895180 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:48.895141 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:49.460149 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:49.460113 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:55.274130 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:55.274093 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:55.377879 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:55.377851 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:15:55.475148 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:15:55.475110 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:05.960763 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:05.960676 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:06.299061 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:06.298957 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:27.291323 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:27.291241 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:33.040234 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:33.040198 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:33.099123 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:33.099095 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:33.381812 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:33.381774 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:33.684511 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:33.684422 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:33.918823 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:33.918787 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:36.884071 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:36.884020 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:37.009703 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:37.009664 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:37.126547 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:37.126515 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:37.385301 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:37.385268 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:37.592139 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:37.592102 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:37.692904 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:37.692819 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:41.360653 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:41.360619 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:41.699908 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:41.699822 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:42.012725 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:42.012625 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:42.874866 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:42.874829 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:44.908953 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:44.908914 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:45.017217 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:45.017173 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:45.117393 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:45.117358 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:45.540548 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:45.540516 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:45.636273 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:45.636239 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:45.737259 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:45.737223 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:49.408403 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:49.408366 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:54.199882 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:54.199842 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:59.196862 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:59.196827 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:16:59.299140 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:16:59.299105 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:17:06.436672 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:17:06.436640 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:17:06.536607 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:17:06.536574 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:17:19.321437 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:17:19.321396 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:17:37.594430 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:17:37.594392 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:17:38.525565 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:17:38.525528 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:17:38.700150 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:17:38.700115 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:17:39.125733 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:17:39.125693 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:17:39.224419 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:17:39.224389 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:17:39.378875 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:17:39.378781 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:17:44.214950 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:17:44.214908 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:20.915199 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:20.915160 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:21.174516 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:21.174419 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:21.562249 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:21.562220 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:21.970247 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:21.970165 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:22.082057 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:22.078570 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:22.178870 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:22.178831 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:22.326181 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:22.326149 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:27.703672 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:27.703636 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:27.759182 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:27.759150 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:27.958985 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:27.958909 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:28.510694 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:28.510653 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:28.817757 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:28.817725 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:28.876887 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:28.876847 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:31.480020 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:31.479985 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:31.576236 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:31.576208 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:31.670686 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:31.670655 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:31.770618 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:31.770525 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:32.007869 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:32.007809 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:32.107934 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:32.107900 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:35.928303 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:35.928271 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:36.373550 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:36.373516 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:36.679285 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:36.679205 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:36.833963 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:36.833930 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:39.474528 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:39.474490 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:39.539714 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:39.539672 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:39.609678 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:39.609639 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:39.747784 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:39.747701 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:40.023672 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:40.023589 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:40.125730 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:40.125702 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:43.948636 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:43.948595 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:47.579195 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:47.579153 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:47.656829 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:47.656793 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:48.646012 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:48.645980 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:48.747601 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:48.747569 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:52.004750 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:52.004714 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:52.405314 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:52.405283 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:52.582681 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:52.582640 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:52.686251 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:52.686168 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:52.835918 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:52.835891 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:58.035258 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:58.035218 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:58.101826 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:58.101791 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:58.290329 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:58.290242 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:58.669357 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:58.669328 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:59.082192 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:59.082154 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:59.180430 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:59.180395 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:59.280624 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:59.280580 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:18:59.366577 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:18:59.366491 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:05.326933 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:05.326899 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:05.427837 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:05.427799 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:05.645264 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:05.645178 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:06.139901 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:06.139864 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:06.541207 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:06.541117 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:06.654429 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:06.654400 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:06.820748 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:06.820712 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:09.675065 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:09.674966 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:09.777908 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:09.777874 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:09.927996 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:09.927881 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:10.030119 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:10.030077 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:10.437002 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:10.436947 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:10.537967 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:10.537929 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:14.168389 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:14.168358 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:15.221687 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:15.221656 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:15.320103 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:15.320074 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:17.720106 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:17.720061 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:17.809314 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:17.809279 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:17.880752 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:17.880717 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:18.279983 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:18.279950 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:18.390262 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:18.390228 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:18.455524 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:18.455483 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:22.231629 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:22.231554 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:31.985927 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:31.985890 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:33.245691 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:33.245618 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:38.969983 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:38.969921 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:19:39.071083 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:19:39.071028 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:20:05.323879 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:20:05.323843 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:20:54.479710 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:20:54.479673 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:21:13.330481 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:21:13.330438 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:21:24.496679 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:21:24.496647 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:11.530317 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:11.530279 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:27.078478 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:27.078396 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:27.467767 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:27.467673 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:27.565267 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:27.565229 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:27.666402 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:27.666369 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:27.750988 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:27.750903 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:33.386052 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:33.385996 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:33.487221 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:33.487187 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:33.711078 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:33.710977 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:34.049664 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:34.049571 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:34.445471 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:34.445441 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:34.596361 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:34.596322 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:34.695221 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:34.695186 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:34.835626 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:34.835594 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:40.299267 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:40.299226 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:40.399393 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:40.399363 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:40.472225 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:40.472194 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:43.885813 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:43.885775 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:45.898762 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:45.898727 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7984469857-gdrg2_8a7afc52-7de8-4884-81b3-0cdab99f23fa/manager/0.log" Jul 08 20:22:46.484335 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:46.484289 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795d769c5f-p5wt4_706248ea-19ee-4702-b43f-957e93191854/manager/0.log" Jul 08 20:22:46.489823 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:46.489799 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795d769c5f-p5wt4_706248ea-19ee-4702-b43f-957e93191854/copy-manifests/0.log" Jul 08 20:22:49.090345 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:49.090309 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-g6bc2_e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2/discovery/0.log" Jul 08 20:22:49.549447 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:49.549372 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-g5476_9ec1fb2b-9530-49e6-98d6-3002e9e0c589/payload-pre-processing/0.log" Jul 08 20:22:49.661211 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:49.661181 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-7rw4p_2350b9d0-b68f-44cd-9bed-954ba64b7685/payload-processing/0.log" Jul 08 20:22:50.111097 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:50.111066 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-5rzck_c4faffbf-2c56-4073-a404-6468008a73b8/main/0.log" Jul 08 20:22:50.328444 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:50.328415 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-cgp7b_b496dbb8-9e0b-4947-9b2f-464dd1b4c47b/main/0.log" Jul 08 20:22:50.438366 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:50.438282 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbm2fwl_88813fd9-c794-4b90-842e-eaf3dc357196/main/0.log" Jul 08 20:22:55.003767 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:55.003731 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:55.104340 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:55.104303 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" limit=1024 Jul 08 20:22:58.005588 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:58.005558 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-82lzg_3184c318-9df8-4f30-92e9-5bf8786149e9/global-pull-secret-syncer/0.log" Jul 08 20:22:58.134859 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:58.134828 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-279bc_33ffe4eb-0ee2-4e34-a03b-b1f06bb98602/konnectivity-agent/0.log" Jul 08 20:22:58.214133 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:22:58.214104 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-198.ec2.internal_4518754363eeef80e023637afe5321de/haproxy/0.log" Jul 08 20:23:04.903786 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:04.903754 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a64384f1-f9ac-4caf-96f7-0fd1fc2722e6/alertmanager/0.log" Jul 08 20:23:04.930546 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:04.930521 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a64384f1-f9ac-4caf-96f7-0fd1fc2722e6/config-reloader/0.log" Jul 08 20:23:04.951238 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:04.951213 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a64384f1-f9ac-4caf-96f7-0fd1fc2722e6/kube-rbac-proxy-web/0.log" Jul 08 20:23:04.976193 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:04.976161 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a64384f1-f9ac-4caf-96f7-0fd1fc2722e6/kube-rbac-proxy/0.log" Jul 08 20:23:05.013061 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.013015 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a64384f1-f9ac-4caf-96f7-0fd1fc2722e6/kube-rbac-proxy-metric/0.log" Jul 08 20:23:05.037939 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.037912 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a64384f1-f9ac-4caf-96f7-0fd1fc2722e6/prom-label-proxy/0.log" Jul 08 20:23:05.059612 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.059589 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a64384f1-f9ac-4caf-96f7-0fd1fc2722e6/init-config-reloader/0.log" Jul 08 20:23:05.104776 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.104742 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-6mr66_c3fcc756-d506-4c6d-b7fc-39cf568fc9e2/cluster-monitoring-operator/0.log" Jul 08 20:23:05.156525 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.156444 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-v5nxp_42d0bbf4-6ee3-4b88-9de0-7316f93170f5/kube-state-metrics/0.log" Jul 08 20:23:05.177187 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.177161 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-v5nxp_42d0bbf4-6ee3-4b88-9de0-7316f93170f5/kube-rbac-proxy-main/0.log" Jul 08 20:23:05.199952 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.199920 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-v5nxp_42d0bbf4-6ee3-4b88-9de0-7316f93170f5/kube-rbac-proxy-self/0.log" Jul 08 20:23:05.226204 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.226175 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5445bcf444-xshpk_ca7c2d56-ffb1-4c71-85cc-62f544725f53/metrics-server/0.log" Jul 08 20:23:05.251167 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.251112 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-cgkzt_7591c324-9865-49fd-80c6-a87d5a0ff352/monitoring-plugin/0.log" Jul 08 20:23:05.280933 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.280905 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j44tg_ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9/node-exporter/0.log" Jul 08 20:23:05.306338 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.306311 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j44tg_ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9/kube-rbac-proxy/0.log" Jul 08 20:23:05.326022 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.325995 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j44tg_ee8c2b98-1b39-4e81-be6e-c42f1f1b20d9/init-textfile/0.log" Jul 08 20:23:05.609649 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.609621 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40c14187-eb76-4a4d-944e-a0f0f7b05bc4/prometheus/0.log" Jul 08 20:23:05.629701 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.629673 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40c14187-eb76-4a4d-944e-a0f0f7b05bc4/config-reloader/0.log" Jul 08 20:23:05.668076 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.668053 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40c14187-eb76-4a4d-944e-a0f0f7b05bc4/thanos-sidecar/0.log" Jul 08 20:23:05.693009 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.692977 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40c14187-eb76-4a4d-944e-a0f0f7b05bc4/kube-rbac-proxy-web/0.log" Jul 08 20:23:05.714932 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.714904 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40c14187-eb76-4a4d-944e-a0f0f7b05bc4/kube-rbac-proxy/0.log" Jul 08 20:23:05.736815 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.736787 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40c14187-eb76-4a4d-944e-a0f0f7b05bc4/kube-rbac-proxy-thanos/0.log" Jul 08 20:23:05.759590 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.759565 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40c14187-eb76-4a4d-944e-a0f0f7b05bc4/init-config-reloader/0.log" Jul 08 20:23:05.789027 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.789001 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-qx8jh_1e3f457a-55de-46ce-8c78-65b0bf1de23d/prometheus-operator/0.log" Jul 08 20:23:05.810810 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.810785 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-qx8jh_1e3f457a-55de-46ce-8c78-65b0bf1de23d/kube-rbac-proxy/0.log" Jul 08 20:23:05.841246 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.841166 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-2sgjx_08a324cd-ac8b-41ff-9300-36f99ae759f1/prometheus-operator-admission-webhook/0.log" Jul 08 20:23:05.873543 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.873510 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-575bcc94c6-fj9zv_b8252151-a32a-4fc4-bb5e-0e9c7c02b07c/telemeter-client/0.log" Jul 08 20:23:05.900937 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.900904 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-575bcc94c6-fj9zv_b8252151-a32a-4fc4-bb5e-0e9c7c02b07c/reload/0.log" Jul 08 20:23:05.922654 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.922628 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-575bcc94c6-fj9zv_b8252151-a32a-4fc4-bb5e-0e9c7c02b07c/kube-rbac-proxy/0.log" Jul 08 20:23:05.952943 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.952914 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d7f878767-smnm7_41a96672-5f5b-4cea-93b9-6a8dbffac896/thanos-query/0.log" Jul 08 20:23:05.974225 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.974199 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d7f878767-smnm7_41a96672-5f5b-4cea-93b9-6a8dbffac896/kube-rbac-proxy-web/0.log" Jul 08 20:23:05.998532 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:05.998510 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d7f878767-smnm7_41a96672-5f5b-4cea-93b9-6a8dbffac896/kube-rbac-proxy/0.log" Jul 08 20:23:06.019618 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:06.019587 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d7f878767-smnm7_41a96672-5f5b-4cea-93b9-6a8dbffac896/prom-label-proxy/0.log" Jul 08 20:23:06.042983 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:06.042946 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d7f878767-smnm7_41a96672-5f5b-4cea-93b9-6a8dbffac896/kube-rbac-proxy-rules/0.log" Jul 08 20:23:06.068493 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:06.068463 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d7f878767-smnm7_41a96672-5f5b-4cea-93b9-6a8dbffac896/kube-rbac-proxy-metrics/0.log" Jul 08 20:23:08.066361 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.066322 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb"] Jul 08 20:23:08.070264 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.070238 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:08.073468 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.073434 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-np6j7\"/\"default-dockercfg-q8fwp\"" Jul 08 20:23:08.074850 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.074829 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-np6j7\"/\"openshift-service-ca.crt\"" Jul 08 20:23:08.074973 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.074883 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-np6j7\"/\"kube-root-ca.crt\"" Jul 08 20:23:08.079156 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.079133 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb"] Jul 08 20:23:08.167734 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.167692 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22cda8f1-f761-40a0-8a86-8895b1aa5a70-sys\") pod \"perf-node-gather-daemonset-sqjzb\" (UID: \"22cda8f1-f761-40a0-8a86-8895b1aa5a70\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:08.167915 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.167772 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8d7c\" (UniqueName: \"kubernetes.io/projected/22cda8f1-f761-40a0-8a86-8895b1aa5a70-kube-api-access-v8d7c\") pod \"perf-node-gather-daemonset-sqjzb\" (UID: \"22cda8f1-f761-40a0-8a86-8895b1aa5a70\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:08.167915 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.167798 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/22cda8f1-f761-40a0-8a86-8895b1aa5a70-podres\") pod \"perf-node-gather-daemonset-sqjzb\" (UID: \"22cda8f1-f761-40a0-8a86-8895b1aa5a70\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:08.167915 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.167823 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/22cda8f1-f761-40a0-8a86-8895b1aa5a70-proc\") pod \"perf-node-gather-daemonset-sqjzb\" (UID: \"22cda8f1-f761-40a0-8a86-8895b1aa5a70\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:08.168021 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.167928 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22cda8f1-f761-40a0-8a86-8895b1aa5a70-lib-modules\") pod \"perf-node-gather-daemonset-sqjzb\" (UID: \"22cda8f1-f761-40a0-8a86-8895b1aa5a70\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:08.268750 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.268714 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8d7c\" (UniqueName: \"kubernetes.io/projected/22cda8f1-f761-40a0-8a86-8895b1aa5a70-kube-api-access-v8d7c\") pod \"perf-node-gather-daemonset-sqjzb\" (UID: \"22cda8f1-f761-40a0-8a86-8895b1aa5a70\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:08.268750 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.268754 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/22cda8f1-f761-40a0-8a86-8895b1aa5a70-podres\") pod \"perf-node-gather-daemonset-sqjzb\" (UID: \"22cda8f1-f761-40a0-8a86-8895b1aa5a70\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:08.268929 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.268782 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/22cda8f1-f761-40a0-8a86-8895b1aa5a70-proc\") pod \"perf-node-gather-daemonset-sqjzb\" (UID: \"22cda8f1-f761-40a0-8a86-8895b1aa5a70\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:08.268929 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.268827 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22cda8f1-f761-40a0-8a86-8895b1aa5a70-lib-modules\") pod \"perf-node-gather-daemonset-sqjzb\" (UID: \"22cda8f1-f761-40a0-8a86-8895b1aa5a70\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:08.268929 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.268867 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22cda8f1-f761-40a0-8a86-8895b1aa5a70-sys\") pod \"perf-node-gather-daemonset-sqjzb\" (UID: \"22cda8f1-f761-40a0-8a86-8895b1aa5a70\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:08.269092 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.268918 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/22cda8f1-f761-40a0-8a86-8895b1aa5a70-proc\") pod \"perf-node-gather-daemonset-sqjzb\" (UID: \"22cda8f1-f761-40a0-8a86-8895b1aa5a70\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:08.269092 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.268936 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22cda8f1-f761-40a0-8a86-8895b1aa5a70-sys\") pod \"perf-node-gather-daemonset-sqjzb\" (UID: \"22cda8f1-f761-40a0-8a86-8895b1aa5a70\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:08.269092 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.268919 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/22cda8f1-f761-40a0-8a86-8895b1aa5a70-podres\") pod \"perf-node-gather-daemonset-sqjzb\" (UID: \"22cda8f1-f761-40a0-8a86-8895b1aa5a70\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:08.269092 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.268956 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22cda8f1-f761-40a0-8a86-8895b1aa5a70-lib-modules\") pod \"perf-node-gather-daemonset-sqjzb\" (UID: \"22cda8f1-f761-40a0-8a86-8895b1aa5a70\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:08.277110 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.277089 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8d7c\" (UniqueName: \"kubernetes.io/projected/22cda8f1-f761-40a0-8a86-8895b1aa5a70-kube-api-access-v8d7c\") pod \"perf-node-gather-daemonset-sqjzb\" (UID: \"22cda8f1-f761-40a0-8a86-8895b1aa5a70\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:08.381575 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.381481 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:08.395463 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.395436 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-79599f8676-4sd6g_86c56241-bacb-4608-abbc-cc356b88ec85/console/0.log" Jul 08 20:23:08.429485 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.429459 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-fsztn_375565ef-c978-4a48-af7f-083f4289557d/download-server/0.log" Jul 08 20:23:08.521378 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.521349 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb"] Jul 08 20:23:08.523476 ip-10-0-129-198 kubenswrapper[2568]: W0708 20:23:08.523446 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod22cda8f1_f761_40a0_8a86_8895b1aa5a70.slice/crio-aacebcd055e2f61fd3c415831f96ed3cd6242270f20feea54442c6a2ddf2e332 WatchSource:0}: Error finding container aacebcd055e2f61fd3c415831f96ed3cd6242270f20feea54442c6a2ddf2e332: Status 404 returned error can't find the container with id aacebcd055e2f61fd3c415831f96ed3cd6242270f20feea54442c6a2ddf2e332 Jul 08 20:23:08.832915 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.832882 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" event={"ID":"22cda8f1-f761-40a0-8a86-8895b1aa5a70","Type":"ContainerStarted","Data":"4b175bf021d7cfd448a36768c962894abda7c33f7673f57f5ca16a31d3f53cd1"} Jul 08 20:23:08.833114 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.832946 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:08.833114 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.832969 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" event={"ID":"22cda8f1-f761-40a0-8a86-8895b1aa5a70","Type":"ContainerStarted","Data":"aacebcd055e2f61fd3c415831f96ed3cd6242270f20feea54442c6a2ddf2e332"} Jul 08 20:23:08.849054 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:08.848980 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" podStartSLOduration=0.84896343 podStartE2EDuration="848.96343ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 20:23:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:23:08.847261986 +0000 UTC m=+2470.540571473" watchObservedRunningTime="2026-07-08 20:23:08.84896343 +0000 UTC m=+2470.542272915" Jul 08 20:23:09.106975 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:09.106949 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-6vp7d_3c0e75a4-b434-4916-894b-19664ea0fd9c/volume-data-source-validator/0.log" Jul 08 20:23:10.081433 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:10.081400 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bvr92_ba1dead2-f97c-4a89-b5d1-487641ab971a/dns/0.log" Jul 08 20:23:10.135658 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:10.135636 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bvr92_ba1dead2-f97c-4a89-b5d1-487641ab971a/kube-rbac-proxy/0.log" Jul 08 20:23:10.355521 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:10.355438 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rz46d_d69db7df-9f0a-4339-99b0-1d7dffa3e30e/dns-node-resolver/0.log" Jul 08 20:23:10.918531 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:10.918505 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-62sgn_a5cf071e-9c63-46a7-8fef-f1f11d2a43cd/node-ca/0.log" Jul 08 20:23:11.968028 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:11.967995 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-g6bc2_e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2/discovery/0.log" Jul 08 20:23:12.116207 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:12.116179 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-g5476_9ec1fb2b-9530-49e6-98d6-3002e9e0c589/payload-pre-processing/0.log" Jul 08 20:23:12.159405 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:12.159369 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-7rw4p_2350b9d0-b68f-44cd-9bed-954ba64b7685/payload-processing/0.log" Jul 08 20:23:12.884075 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:12.884030 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vppxs_faf3ad2e-6304-49c5-be6e-560a500240f8/serve-healthcheck-canary/0.log" Jul 08 20:23:13.342736 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:13.342704 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6qk8p_a0479c7a-9963-4c7d-be2b-2860133eb4a4/kube-rbac-proxy/0.log" Jul 08 20:23:13.364162 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:13.364136 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6qk8p_a0479c7a-9963-4c7d-be2b-2860133eb4a4/exporter/0.log" Jul 08 20:23:13.386864 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:13.386840 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6qk8p_a0479c7a-9963-4c7d-be2b-2860133eb4a4/extractor/0.log" Jul 08 20:23:14.845469 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:14.845442 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-sqjzb" Jul 08 20:23:15.757200 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:15.757162 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7984469857-gdrg2_8a7afc52-7de8-4884-81b3-0cdab99f23fa/manager/0.log" Jul 08 20:23:16.026962 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:16.026881 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795d769c5f-p5wt4_706248ea-19ee-4702-b43f-957e93191854/manager/0.log" Jul 08 20:23:16.048173 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:16.048147 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795d769c5f-p5wt4_706248ea-19ee-4702-b43f-957e93191854/copy-manifests/0.log" Jul 08 20:23:17.376023 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:17.375992 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7cffbd996-mjwfz_023f102d-9467-43d7-862d-57e07a82f63a/manager/0.log" Jul 08 20:23:18.444489 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:18.444456 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-g6bc2_e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2/discovery/0.log" Jul 08 20:23:18.547666 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:18.547636 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-g5476_9ec1fb2b-9530-49e6-98d6-3002e9e0c589/payload-pre-processing/0.log" Jul 08 20:23:18.583634 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:18.583602 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-7rw4p_2350b9d0-b68f-44cd-9bed-954ba64b7685/payload-processing/0.log" Jul 08 20:23:19.841427 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:19.841398 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-g6bc2_e6709fe0-a57b-4f7f-8d1b-e2b764ff9ac2/discovery/0.log" Jul 08 20:23:19.953787 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:19.953752 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-g5476_9ec1fb2b-9530-49e6-98d6-3002e9e0c589/payload-pre-processing/0.log" Jul 08 20:23:19.990643 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:19.990610 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-7rw4p_2350b9d0-b68f-44cd-9bed-954ba64b7685/payload-processing/0.log" Jul 08 20:23:22.485621 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:22.485587 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-6n58q_d95cfbc4-4f29-4b28-b37e-7869239b6302/migrator/0.log" Jul 08 20:23:22.504589 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:22.504569 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-6n58q_d95cfbc4-4f29-4b28-b37e-7869239b6302/graceful-termination/0.log" Jul 08 20:23:23.935069 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:23.935025 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g5gnj_d277414e-55a2-411f-852d-aae01918e1c4/kube-multus-additional-cni-plugins/0.log" Jul 08 20:23:23.955569 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:23.955539 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g5gnj_d277414e-55a2-411f-852d-aae01918e1c4/egress-router-binary-copy/0.log" Jul 08 20:23:23.975823 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:23.975797 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g5gnj_d277414e-55a2-411f-852d-aae01918e1c4/cni-plugins/0.log" Jul 08 20:23:23.995166 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:23.995136 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g5gnj_d277414e-55a2-411f-852d-aae01918e1c4/bond-cni-plugin/0.log" Jul 08 20:23:24.015290 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:24.015263 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g5gnj_d277414e-55a2-411f-852d-aae01918e1c4/routeoverride-cni/0.log" Jul 08 20:23:24.034834 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:24.034806 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g5gnj_d277414e-55a2-411f-852d-aae01918e1c4/whereabouts-cni-bincopy/0.log" Jul 08 20:23:24.054827 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:24.054802 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g5gnj_d277414e-55a2-411f-852d-aae01918e1c4/whereabouts-cni/0.log" Jul 08 20:23:24.272289 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:24.272264 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-h77hk_baeda2ff-0e38-4c4a-a057-dcd4e9020371/kube-multus/0.log" Jul 08 20:23:24.407208 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:24.407176 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-s7hbd_3b5311a0-d5fa-4994-9071-d861d2a53b09/network-metrics-daemon/0.log" Jul 08 20:23:24.425247 ip-10-0-129-198 kubenswrapper[2568]: I0708 20:23:24.425218 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-s7hbd_3b5311a0-d5fa-4994-9071-d861d2a53b09/kube-rbac-proxy/0.log"