Jul 13 18:58:34.404048 ip-10-0-128-30 systemd[1]: Starting Kubernetes Kubelet... Jul 13 18:58:34.887304 ip-10-0-128-30 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 18:58:34.887304 ip-10-0-128-30 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 18:58:34.887304 ip-10-0-128-30 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 18:58:34.887304 ip-10-0-128-30 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 18:58:34.887304 ip-10-0-128-30 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 18:58:34.887304 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.798690 2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 18:58:34.887304 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804387 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 18:58:34.887304 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804401 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 18:58:34.887304 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804404 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 18:58:34.887304 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804415 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 18:58:34.887304 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804418 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 18:58:34.887304 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804421 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 18:58:34.887304 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804424 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 18:58:34.887304 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804427 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 18:58:34.866574 ip-10-0-128-30 systemd[1]: Started Kubernetes Kubelet. Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804430 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804432 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804435 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804438 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804440 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804443 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804446 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804448 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804451 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804458 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804461 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804463 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804466 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804469 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804471 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804473 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804476 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804478 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804481 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 18:58:34.888848 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804485 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804487 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804492 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804496 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804499 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804502 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804505 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804508 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804511 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804513 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804522 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804524 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804527 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804529 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804532 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804534 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804537 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804539 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804542 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804545 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 18:58:34.890069 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804548 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804552 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804555 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804558 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804560 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804563 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804566 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804569 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804572 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804575 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804578 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804580 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804583 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804585 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804588 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804591 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804593 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804596 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804598 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 18:58:34.891130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804601 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804603 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804606 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804610 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804619 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804622 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804625 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804627 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804630 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804634 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804636 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804639 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804641 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804644 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804647 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804649 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804652 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804655 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804657 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804660 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 18:58:34.892223 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804662 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804665 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804668 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804671 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804673 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804676 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804678 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804681 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804684 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804686 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804688 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804691 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.804693 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805507 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805515 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805518 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805521 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805530 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805533 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 18:58:34.893319 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805536 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805539 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805541 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805544 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805546 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805549 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805553 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805556 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805559 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805562 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805565 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805568 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805570 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805573 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805576 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805578 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805581 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805583 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805586 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805588 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 18:58:34.894391 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805591 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805594 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805596 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805599 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805601 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805604 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805607 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805611 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805614 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805617 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805621 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805630 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805633 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805635 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805638 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805640 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805642 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805645 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805647 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 18:58:34.895458 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805651 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805654 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805657 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805659 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805662 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805664 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805667 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805669 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805672 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805674 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805677 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805679 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805682 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805684 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805687 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805689 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805692 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805694 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805697 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805699 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 18:58:34.896309 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805702 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805704 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805707 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805710 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805712 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805720 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805723 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805726 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805729 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805731 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805734 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805736 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805739 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805741 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805744 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805747 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805749 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805751 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805754 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805757 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 18:58:34.897416 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805760 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805762 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805764 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805767 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805769 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805772 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805774 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805777 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805793 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805796 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805799 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805802 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805805 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.805810 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808227 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808238 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808242 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808245 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808248 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808255 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 18:58:34.898618 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808258 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808261 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808265 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808268 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808270 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808273 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808277 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808283 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808285 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808288 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808292 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808299 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808302 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808305 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808308 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808311 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808314 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808323 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808327 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808329 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 18:58:34.899495 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808332 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808335 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808338 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808343 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808346 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808349 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808352 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808355 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808357 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808360 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808363 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808366 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808368 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808371 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808373 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808376 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808381 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808384 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808387 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808389 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 18:58:34.903719 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808392 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808395 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808398 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808400 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808403 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808407 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808411 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808417 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808420 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808423 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808433 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808437 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808440 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808443 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808446 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808449 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808452 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808455 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808458 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 18:58:34.904593 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808460 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808466 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808468 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808471 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808474 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808482 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808487 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808491 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808495 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808497 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808500 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808503 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808509 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808512 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808515 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808518 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808521 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808524 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808527 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 18:58:34.908234 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808530 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808533 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808536 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808538 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808541 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808549 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808555 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808557 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808560 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808563 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808568 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808571 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808573 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808576 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.808579 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808678 2574 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808686 2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808696 2574 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808700 2574 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808705 2574 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808708 2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 18:58:34.908883 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808720 2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808725 2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808728 2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808731 2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808735 2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808738 2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808741 2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808745 2574 flags.go:64] FLAG: --cgroup-root="" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808752 2574 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808759 2574 flags.go:64] FLAG: --client-ca-file="" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808763 2574 flags.go:64] FLAG: --cloud-provider="external" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808767 2574 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808774 2574 flags.go:64] FLAG: --cluster-domain="" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808777 2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808796 2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808800 2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808806 2574 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808817 2574 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808820 2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808823 2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808827 2574 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808834 2574 flags.go:64] FLAG: --contention-profiling="false" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808837 2574 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808840 2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 18:58:35.153156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808846 2574 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808851 2574 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808855 2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808859 2574 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808862 2574 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808865 2574 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808868 2574 flags.go:64] FLAG: --enable-server="true" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808871 2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808879 2574 flags.go:64] FLAG: --event-burst="100" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808885 2574 flags.go:64] FLAG: --event-qps="50" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808888 2574 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808891 2574 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808895 2574 flags.go:64] FLAG: --eviction-hard="" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808899 2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808902 2574 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808905 2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808908 2574 flags.go:64] FLAG: --eviction-soft="" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808918 2574 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808921 2574 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808924 2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808927 2574 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808930 2574 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808934 2574 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808937 2574 flags.go:64] FLAG: --feature-gates="" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808941 2574 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 18:58:35.167643 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808947 2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808963 2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808966 2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808970 2574 flags.go:64] FLAG: --healthz-port="10248" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808974 2574 flags.go:64] FLAG: --help="false" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808977 2574 flags.go:64] FLAG: --hostname-override="ip-10-0-128-30.ec2.internal" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808981 2574 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808984 2574 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808989 2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808995 2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.808999 2574 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809002 2574 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809005 2574 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809008 2574 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809011 2574 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809014 2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809017 2574 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809022 2574 flags.go:64] FLAG: --kube-reserved="" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809026 2574 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809029 2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809032 2574 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809035 2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809038 2574 flags.go:64] FLAG: --lock-file="" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809041 2574 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 18:58:35.169247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809044 2574 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809047 2574 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809055 2574 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809058 2574 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809060 2574 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809063 2574 flags.go:64] FLAG: --logging-format="text" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809066 2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809070 2574 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809073 2574 flags.go:64] FLAG: --manifest-url="" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809076 2574 flags.go:64] FLAG: --manifest-url-header="" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809089 2574 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809093 2574 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809098 2574 flags.go:64] FLAG: --max-pods="110" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809101 2574 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809105 2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809107 2574 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809112 2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809115 2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809118 2574 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809123 2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809131 2574 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809134 2574 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809144 2574 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 18:58:35.170427 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809149 2574 flags.go:64] FLAG: --pod-cidr="" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809152 2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809158 2574 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809164 2574 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809167 2574 flags.go:64] FLAG: --pods-per-core="0" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809170 2574 flags.go:64] FLAG: --port="10250" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809173 2574 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809176 2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09cea1030ca59c35d" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809180 2574 flags.go:64] FLAG: --qos-reserved="" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809183 2574 flags.go:64] FLAG: --read-only-port="10255" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809186 2574 flags.go:64] FLAG: --register-node="true" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809191 2574 flags.go:64] FLAG: --register-with-taints="" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809195 2574 flags.go:64] FLAG: --registry-burst="10" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809198 2574 flags.go:64] FLAG: --registry-qps="5" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809201 2574 flags.go:64] FLAG: --reserved-cpus="" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809204 2574 flags.go:64] FLAG: --reserved-memory="" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809208 2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809211 2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809214 2574 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809217 2574 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809236 2574 flags.go:64] FLAG: --runonce="false" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809239 2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809244 2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809247 2574 flags.go:64] FLAG: --seccomp-default="false" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809250 2574 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 18:58:35.171369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809254 2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809258 2574 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809261 2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809266 2574 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809269 2574 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809272 2574 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809275 2574 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809278 2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809282 2574 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809285 2574 flags.go:64] FLAG: --system-cgroups="" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809288 2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809297 2574 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809300 2574 flags.go:64] FLAG: --tls-cert-file="" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809303 2574 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809310 2574 flags.go:64] FLAG: --tls-min-version="" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809312 2574 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809315 2574 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809318 2574 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809324 2574 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809327 2574 flags.go:64] FLAG: --v="2" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809331 2574 flags.go:64] FLAG: --version="false" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809335 2574 flags.go:64] FLAG: --vmodule="" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809339 2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.809342 2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809621 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 18:58:35.172555 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809625 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809628 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809632 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809641 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809645 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809651 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809654 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809658 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809661 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809664 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809667 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809669 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809672 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809675 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809678 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809681 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809684 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809686 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809692 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809695 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 18:58:35.173546 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809697 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809700 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809703 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809706 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809712 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809715 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809718 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809720 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809723 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809726 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809731 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809734 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809737 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809740 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809758 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809860 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809870 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809874 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809877 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 18:58:35.174547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809881 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809884 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809887 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809890 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809892 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809895 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809898 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809900 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809903 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809906 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809908 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809911 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809913 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809916 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809918 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809921 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809923 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809926 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809929 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809931 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 18:58:35.175344 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809934 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809936 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809939 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809941 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809944 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809946 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809949 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809951 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809954 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809958 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809960 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809963 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809967 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809970 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809973 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809977 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809981 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809983 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809986 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809989 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 18:58:35.176334 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809991 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 18:58:35.177139 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809994 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 18:58:35.177139 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809996 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 18:58:35.177139 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.809999 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 18:58:35.177139 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.810002 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 18:58:35.177139 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.810004 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 18:58:35.177139 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.810007 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 18:58:35.177139 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.810009 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 18:58:35.177139 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.810012 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 18:58:35.177139 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.810018 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 18:58:35.177139 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.810022 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 18:58:35.177139 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.810027 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 18:58:35.177139 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.810032 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 18:58:35.177139 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.810035 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 18:58:35.177139 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.810037 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 18:58:35.177139 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.810040 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 18:58:35.177139 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.810043 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 18:58:35.177139 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.810051 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 18:58:35.177139 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.810054 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 18:58:35.177938 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.810737 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 18:58:35.177938 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.817710 2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 18:58:35.177938 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.817727 2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 18:58:35.177938 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.817750 2574 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 18:58:35.177938 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.817756 2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 18:58:35.177938 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817833 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 18:58:35.177938 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817840 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 18:58:35.177938 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817843 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 18:58:35.177938 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817846 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 18:58:35.177938 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817849 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 18:58:35.177938 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817852 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 18:58:35.177938 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817855 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 18:58:35.177938 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817859 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 18:58:35.177938 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817862 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 18:58:35.177938 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817864 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817867 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817870 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817874 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817877 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817880 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817883 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817886 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817889 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817892 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817896 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817900 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817903 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817906 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817908 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817911 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817915 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817918 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817921 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 18:58:35.178539 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817924 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817927 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817929 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817932 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817934 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817937 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817940 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817942 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817945 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817948 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817954 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817956 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817959 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817962 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817964 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817967 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817970 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817972 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817975 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817978 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 18:58:35.179335 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817981 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817983 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817986 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817989 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817992 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817994 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817997 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.817999 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818002 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818005 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818008 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818010 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818013 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818015 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818018 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818020 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818023 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818025 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818028 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 18:58:35.180130 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818030 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818034 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818038 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818043 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818046 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818049 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818052 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818055 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818057 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818060 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818063 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818065 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818069 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818072 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818074 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818077 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818079 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818082 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818085 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 18:58:35.180933 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818087 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 18:58:35.181742 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818089 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 18:58:35.181742 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818092 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 18:58:35.181742 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818095 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 18:58:35.181742 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818097 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 18:58:35.181742 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818101 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 18:58:35.181742 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818103 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 18:58:35.181742 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818106 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 18:58:35.181742 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818108 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 18:58:35.181742 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818111 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 18:58:35.181742 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818113 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 18:58:35.181742 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818116 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 18:58:35.181742 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818119 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 18:58:35.181742 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.818124 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 18:58:35.181742 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818241 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 18:58:35.181742 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818245 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818248 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818252 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818255 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818257 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818260 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818263 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818265 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818269 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818272 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818276 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818279 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818282 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818284 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818287 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818289 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818292 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818295 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818297 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 18:58:35.182547 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818300 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818303 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818305 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818308 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818311 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818314 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818317 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818319 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818322 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818324 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818327 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818329 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818332 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818334 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818337 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818341 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818344 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818346 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818349 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818351 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818354 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 18:58:35.185430 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818357 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818360 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818363 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818366 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818368 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818371 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818373 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818376 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818378 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818381 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818383 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818385 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818388 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818391 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818393 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818395 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818400 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818403 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818406 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818409 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 18:58:35.186412 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818412 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818414 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818417 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818419 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818422 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818425 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818428 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818431 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818433 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818436 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818440 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818443 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818446 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818448 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818452 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818455 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818458 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818460 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818463 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 18:58:35.189537 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818465 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 18:58:35.190080 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818468 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 18:58:35.190080 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818470 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 18:58:35.190080 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818476 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 18:58:35.190080 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818478 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 18:58:35.190080 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818481 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 18:58:35.190080 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818484 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 18:58:35.190080 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818486 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 18:58:35.190080 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818489 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 18:58:35.190080 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818491 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 18:58:35.190080 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818494 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 18:58:35.190080 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818497 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 18:58:35.190080 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818500 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 18:58:35.190080 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818502 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 18:58:35.190080 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818504 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 18:58:35.190080 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818507 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 18:58:35.190080 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818509 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 18:58:35.190080 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818512 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 18:58:35.190080 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:34.818514 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 18:58:35.417211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.818520 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 18:58:35.417211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.818642 2574 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 18:58:35.417211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.820743 2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 18:58:35.417211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.822205 2574 server.go:1019] "Starting client certificate rotation" Jul 13 18:58:35.417211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.822308 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 18:58:35.417211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.822347 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 18:58:35.417211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.824763 2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 18:58:35.417211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.826680 2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 18:58:35.417211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.837421 2574 log.go:25] "Validated CRI v1 runtime API" Jul 13 18:58:35.417211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.839540 2574 log.go:25] "Validated CRI v1 image API" Jul 13 18:58:35.417211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.840291 2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 18:58:35.417211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.843699 2574 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 e2b6c7b3-daad-4110-8ce3-96537af9524a:/dev/nvme0n1p4 edcd6adb-fd9e-4f1d-a711-9238d098817e:/dev/nvme0n1p3] Jul 13 18:58:35.418609 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.843734 2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 18:58:35.418609 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.848869 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 18:58:35.419002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.849222 2574 manager.go:216] Machine: {Timestamp:2026-07-13 18:58:34.847945659 +0000 UTC m=+0.324399825 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098591 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2736badde92a6e2c6fa44b8346a196 SystemUUID:ec2736ba-dde9-2a6e-2c6f-a44b8346a196 BootID:4330a72f-67d5-48f1-bf7d-37f7714ba9ae Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6d:44:a6:40:2f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6d:44:a6:40:2f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:ae:7f:57:da:9b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 18:58:35.419002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.849332 2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 18:58:35.419002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.849427 2574 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 18:58:35.419002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.851052 2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 18:58:35.419002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.851078 2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-30.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 18:58:35.419002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.851725 2574 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 18:58:35.419002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.851735 2574 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 18:58:35.419002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.851743 2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 18:58:35.419002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.852474 2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 18:58:35.419002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.853817 2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 18:58:35.419002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.856607 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 13 18:58:35.419002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.856728 2574 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 18:58:35.419002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.856759 2574 kubelet.go:486] "Attempting to sync node with API server" Jul 13 18:58:35.419002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.856771 2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 18:58:35.419002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.856801 2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 18:58:35.419002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.856815 2574 kubelet.go:398] "Adding apiserver pod source" Jul 13 18:58:35.419002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.856827 2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 18:58:35.419002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.857261 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.857278 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.859436 2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.861282 2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.861310 2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.861467 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.861479 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.861485 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.861491 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.861499 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.861504 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.861954 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.861961 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.861969 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.861974 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.861990 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.862000 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.862712 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.862719 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.865720 2574 server.go:1268] "Started kubelet" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.865811 2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.865841 2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.865877 2574 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.866985 2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.869714 2574 server.go:310] "Adding debug handlers to kubelet server" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.870339 2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-30.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:34.871423 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-30.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:34.871503 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:34.870773 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-30.ec2.internal.18c1ee7545d676f2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-30.ec2.internal,UID:ip-10-0-128-30.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-30.ec2.internal,},FirstTimestamp:2026-07-13 18:58:34.86569445 +0000 UTC m=+0.342148614,LastTimestamp:2026-07-13 18:58:34.86569445 +0000 UTC m=+0.342148614,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-30.ec2.internal,}" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:34.871910 2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.872442 2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.872833 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.878348 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-222cg" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:34.878419 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.879522 2574 factory.go:55] Registering systemd factory Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.879543 2574 factory.go:223] Registration of the systemd container factory successfully Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.879758 2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.879861 2574 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.879878 2574 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.879919 2574 factory.go:153] Registering CRI-O factory Jul 13 18:58:35.419828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.879931 2574 factory.go:223] Registration of the crio container factory successfully Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.880002 2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.880023 2574 reconstruct.go:97] "Volume reconstruction finished" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.880034 2574 reconciler.go:29] "Reconciler: start to sync state" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.880024 2574 factory.go:103] Registering Raw factory Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.880068 2574 manager.go:1196] Started watching for new ooms in manager Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.880492 2574 manager.go:349] Starting recovery of all containers Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.882021 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-222cg" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:34.884585 2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-30.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:34.884724 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.891998 2574 manager.go:354] Recovery completed Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.897019 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.899083 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.899111 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.899124 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.899674 2574 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.899686 2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.899688 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.899704 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.900461 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.900494 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.900507 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.900860 2574 policy_none.go:49] "None policy: Start" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.900873 2574 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.900882 2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:34.901235 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-30.ec2.internal.18c1ee7547d4253d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-30.ec2.internal,UID:ip-10-0-128-30.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-30.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-30.ec2.internal,},FirstTimestamp:2026-07-13 18:58:34.899096893 +0000 UTC m=+0.375551060,LastTimestamp:2026-07-13 18:58:34.899096893 +0000 UTC m=+0.375551060,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-30.ec2.internal,}" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.901901 2574 policy_none.go:47] "Start" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.932458 2574 manager.go:337] "Starting Device Plugin manager" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:34.932500 2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.932511 2574 server.go:87] "Starting device plugin registration server" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.932756 2574 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.932767 2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.932893 2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.932987 2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:34.932997 2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:34.940104 2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:34.940143 2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-30.ec2.internal\" not found" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.034998 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.036060 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:58:35.421404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.036089 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.036099 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.036141 2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-30.ec2.internal" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.044826 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.047951 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.047983 2574 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.048004 2574 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:35.048038 2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.051552 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.062153 2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-30.ec2.internal" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:35.062172 2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-30.ec2.internal\": node \"ip-10-0-128-30.ec2.internal\" not found" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.149053 2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal"] Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.149126 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.155375 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.155395 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.155405 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.156921 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.157093 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.157555 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.159287 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.159307 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.159316 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.159287 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.159391 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.159404 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.160643 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.160665 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.161208 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.161228 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.161241 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:35.162573 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.181210 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2a9da3169af3e3a8e9cc39dc3e8dbe13-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal\" (UID: \"2a9da3169af3e3a8e9cc39dc3e8dbe13\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.181233 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a9da3169af3e3a8e9cc39dc3e8dbe13-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal\" (UID: \"2a9da3169af3e3a8e9cc39dc3e8dbe13\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.181248 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f50767bdb9b137e0ddb50e76b305359-config\") pod \"kube-apiserver-proxy-ip-10-0-128-30.ec2.internal\" (UID: \"9f50767bdb9b137e0ddb50e76b305359\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:35.207553 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-30.ec2.internal\" not found" node="ip-10-0-128-30.ec2.internal" Jul 13 18:58:35.423016 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:35.212926 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-30.ec2.internal\" not found" node="ip-10-0-128-30.ec2.internal" Jul 13 18:58:35.424420 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:35.263194 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found" Jul 13 18:58:35.424420 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.281306 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2a9da3169af3e3a8e9cc39dc3e8dbe13-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal\" (UID: \"2a9da3169af3e3a8e9cc39dc3e8dbe13\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" Jul 13 18:58:35.424420 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.281332 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a9da3169af3e3a8e9cc39dc3e8dbe13-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal\" (UID: \"2a9da3169af3e3a8e9cc39dc3e8dbe13\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" Jul 13 18:58:35.424420 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.281347 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f50767bdb9b137e0ddb50e76b305359-config\") pod \"kube-apiserver-proxy-ip-10-0-128-30.ec2.internal\" (UID: \"9f50767bdb9b137e0ddb50e76b305359\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal" Jul 13 18:58:35.424420 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.281381 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f50767bdb9b137e0ddb50e76b305359-config\") pod \"kube-apiserver-proxy-ip-10-0-128-30.ec2.internal\" (UID: \"9f50767bdb9b137e0ddb50e76b305359\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal" Jul 13 18:58:35.424420 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.281402 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2a9da3169af3e3a8e9cc39dc3e8dbe13-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal\" (UID: \"2a9da3169af3e3a8e9cc39dc3e8dbe13\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" Jul 13 18:58:35.424420 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.281414 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a9da3169af3e3a8e9cc39dc3e8dbe13-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal\" (UID: \"2a9da3169af3e3a8e9cc39dc3e8dbe13\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" Jul 13 18:58:35.424420 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:35.363913 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found" Jul 13 18:58:35.683086 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:35.464330 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found" Jul 13 18:58:35.683086 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.509796 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" Jul 13 18:58:35.683086 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.515236 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal" Jul 13 18:58:35.683086 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:35.565523 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found" Jul 13 18:58:35.683086 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:35.665996 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found" Jul 13 18:58:35.913360 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:35.766493 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found" Jul 13 18:58:35.913360 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.821966 2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 18:58:35.913360 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.822119 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 18:58:35.913360 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:35.867406 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found" Jul 13 18:58:35.913360 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.891444 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 18:53:34 +0000 UTC" deadline="2028-02-23 18:57:12.892485282 +0000 UTC" Jul 13 18:58:35.913360 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:35.891467 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14159h58m37.001021323s" Jul 13 18:58:36.166795 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:35.967661 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found" Jul 13 18:58:36.166795 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:36.067962 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found" Jul 13 18:58:36.168562 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:36.168263 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found" Jul 13 18:58:36.363885 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.195618 2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 18:58:36.363885 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:36.200428 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f50767bdb9b137e0ddb50e76b305359.slice/crio-b5253404c2d273e34b940907b3db94c1bfc3f717a902e76ddb6c53439a35b45a WatchSource:0}: Error finding container b5253404c2d273e34b940907b3db94c1bfc3f717a902e76ddb6c53439a35b45a: Status 404 returned error can't find the container with id b5253404c2d273e34b940907b3db94c1bfc3f717a902e76ddb6c53439a35b45a Jul 13 18:58:36.363885 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:36.200621 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a9da3169af3e3a8e9cc39dc3e8dbe13.slice/crio-755abd8ef01f04c471a9c4afecb97f727235160959dc4144db92782359d8b82c WatchSource:0}: Error finding container 755abd8ef01f04c471a9c4afecb97f727235160959dc4144db92782359d8b82c: Status 404 returned error can't find the container with id 755abd8ef01f04c471a9c4afecb97f727235160959dc4144db92782359d8b82c Jul 13 18:58:36.363885 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.206675 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 18:58:36.363885 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:36.269256 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found" Jul 13 18:58:36.370747 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:36.369932 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found" Jul 13 18:58:36.617363 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.427586 2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 18:58:36.617363 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.443107 2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 18:58:36.617363 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.478681 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" Jul 13 18:58:36.617363 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.491441 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 18:58:36.617363 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.492340 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal" Jul 13 18:58:36.617363 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.503232 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 18:58:36.836114 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.834640 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.857489 2574 apiserver.go:52] "Watching apiserver" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.863968 2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.864394 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-8v8bj","kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw","openshift-cluster-node-tuning-operator/tuned-slbqz","openshift-multus/multus-4pbnd","openshift-multus/multus-additional-cni-plugins-qgq48","openshift-network-operator/iptables-alerter-x6wbt","openshift-ovn-kubernetes/ovnkube-node-79lb6","kube-system/global-pull-secret-syncer-5ftlh","openshift-dns/node-resolver-jczlv","openshift-image-registry/node-ca-xdbcs","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal","openshift-multus/network-metrics-daemon-26wx8","openshift-network-diagnostics/network-check-target-f4ffp"] Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.866360 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x6wbt" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.868020 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:36.868105 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.868619 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.868658 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.868684 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hph7t\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.868690 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.870398 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.872982 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.873231 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fs6zk\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.873449 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.873461 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.873745 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.876498 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.876515 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lsnbz\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.876662 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.877283 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4pbnd" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.878015 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.878507 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.878892 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.879179 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.879518 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.879683 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lk2dz\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.880028 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.880157 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.880357 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kgz75\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.880594 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 18:58:36.922128 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.881741 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:36.881814 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.882121 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8v8bj" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.883912 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.884213 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.884381 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5rpcd\"" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.884540 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:36.884585 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.886344 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jczlv" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.887964 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.888753 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fmmcw\"" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.888919 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889265 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-dbus\") pod \"global-pull-secret-syncer-5ftlh\" (UID: \"e8bde243-4f0c-4fd7-a18c-b5900c1609e8\") " pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889291 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-sysconfig\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889312 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-cni-binary-copy\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889334 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b5a54297-cff5-4224-b5c8-6e1f059d5580-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889358 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs\") pod \"network-metrics-daemon-26wx8\" (UID: \"b735dd6a-c571-43a2-a6f7-56a9094519a9\") " pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889378 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/763b955f-d72c-46af-8046-87d4b90b7f12-socket-dir\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889398 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/763b955f-d72c-46af-8046-87d4b90b7f12-device-dir\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889418 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42a54609-e09f-4ded-a4c8-c35f6b6365c9-host-slash\") pod \"iptables-alerter-x6wbt\" (UID: \"42a54609-e09f-4ded-a4c8-c35f6b6365c9\") " pod="openshift-network-operator/iptables-alerter-x6wbt" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889454 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-host-var-lib-cni-bin\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:36.924477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889472 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret\") pod \"global-pull-secret-syncer-5ftlh\" (UID: \"e8bde243-4f0c-4fd7-a18c-b5900c1609e8\") " pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:36.926232 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889493 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-system-cni-dir\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:36.926232 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889516 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-host-var-lib-cni-multus\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:36.926232 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889531 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nbrd\" (UniqueName: \"kubernetes.io/projected/b5a54297-cff5-4224-b5c8-6e1f059d5580-kube-api-access-2nbrd\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:36.926232 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889551 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-modprobe-d\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:36.926232 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889570 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-tuned\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:36.926232 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889591 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5a54297-cff5-4224-b5c8-6e1f059d5580-cni-binary-copy\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:36.926232 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889611 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5a54297-cff5-4224-b5c8-6e1f059d5580-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:36.926232 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889642 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5a54297-cff5-4224-b5c8-6e1f059d5580-cnibin\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:36.926232 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889662 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/763b955f-d72c-46af-8046-87d4b90b7f12-sys-fs\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:36.926232 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889682 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-kubernetes\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:36.926232 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889711 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-host-run-k8s-cni-cncf-io\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:36.926232 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889745 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/763b955f-d72c-46af-8046-87d4b90b7f12-registration-dir\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:36.926232 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889766 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-662nw\" (UniqueName: \"kubernetes.io/projected/f28fb900-3954-4e4e-a7a1-527ea570ae72-kube-api-access-662nw\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:36.926232 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889814 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpd78\" (UniqueName: \"kubernetes.io/projected/b735dd6a-c571-43a2-a6f7-56a9094519a9-kube-api-access-qpd78\") pod \"network-metrics-daemon-26wx8\" (UID: \"b735dd6a-c571-43a2-a6f7-56a9094519a9\") " pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:36.926232 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889843 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-multus-daemon-config\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:36.926232 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889864 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/763b955f-d72c-46af-8046-87d4b90b7f12-etc-selinux\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:36.927408 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889884 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-sys\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:36.927408 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889904 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-multus-conf-dir\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:36.927408 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889918 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b5a54297-cff5-4224-b5c8-6e1f059d5580-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:36.927408 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889936 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-sysctl-d\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:36.927408 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889951 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-run\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:36.927408 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889963 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-lib-modules\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:36.927408 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889977 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj25s\" (UniqueName: \"kubernetes.io/projected/42a54609-e09f-4ded-a4c8-c35f6b6365c9-kube-api-access-mj25s\") pod \"iptables-alerter-x6wbt\" (UID: \"42a54609-e09f-4ded-a4c8-c35f6b6365c9\") " pod="openshift-network-operator/iptables-alerter-x6wbt" Jul 13 18:58:36.927408 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.889998 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-multus-cni-dir\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:36.927408 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890022 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/763b955f-d72c-46af-8046-87d4b90b7f12-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:36.927408 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890054 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x544r\" (UniqueName: \"kubernetes.io/projected/763b955f-d72c-46af-8046-87d4b90b7f12-kube-api-access-x544r\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:36.927408 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890076 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/42a54609-e09f-4ded-a4c8-c35f6b6365c9-iptables-alerter-script\") pod \"iptables-alerter-x6wbt\" (UID: \"42a54609-e09f-4ded-a4c8-c35f6b6365c9\") " pod="openshift-network-operator/iptables-alerter-x6wbt" Jul 13 18:58:36.927408 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890097 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-os-release\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:36.927408 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890110 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f079acce-c9cf-4ba0-b003-a176d76fa3ad-agent-certs\") pod \"konnectivity-agent-8v8bj\" (UID: \"f079acce-c9cf-4ba0-b003-a176d76fa3ad\") " pod="kube-system/konnectivity-agent-8v8bj" Jul 13 18:58:36.927408 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890123 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f079acce-c9cf-4ba0-b003-a176d76fa3ad-konnectivity-ca\") pod \"konnectivity-agent-8v8bj\" (UID: \"f079acce-c9cf-4ba0-b003-a176d76fa3ad\") " pod="kube-system/konnectivity-agent-8v8bj" Jul 13 18:58:36.927408 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890140 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-host-run-multus-certs\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:36.927408 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890155 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b5a54297-cff5-4224-b5c8-6e1f059d5580-os-release\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890177 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-kubelet-config\") pod \"global-pull-secret-syncer-5ftlh\" (UID: \"e8bde243-4f0c-4fd7-a18c-b5900c1609e8\") " pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890234 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-host\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890251 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-cnibin\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890267 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-hostroot\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890282 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-etc-kubernetes\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890296 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5a54297-cff5-4224-b5c8-6e1f059d5580-system-cni-dir\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890310 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnjkj\" (UniqueName: \"kubernetes.io/projected/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-kube-api-access-cnjkj\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890324 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-sysctl-conf\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890338 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f28fb900-3954-4e4e-a7a1-527ea570ae72-tmp\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890351 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-multus-socket-dir-parent\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890454 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-host-run-netns\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890477 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-systemd\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890498 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-var-lib-kubelet\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.890519 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-host-var-lib-kubelet\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.891154 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.892305 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xdbcs" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.892685 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.894493 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.894837 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-g6n5d\"" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.894972 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 18:58:36.938134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.895114 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 18:58:36.939494 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.895236 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 18:58:36.939494 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.895517 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8k96r\"" Jul 13 18:58:36.939494 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.896071 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 18:58:36.939494 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.896105 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 18:58:36.939494 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.896233 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 18:58:36.939494 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.896283 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 18:58:36.939494 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.896440 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 18:58:36.939494 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.914014 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zl9dw" Jul 13 18:58:36.939494 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.921489 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zl9dw" Jul 13 18:58:37.030265 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.981716 2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 18:58:37.030265 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991103 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-run-netns\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.030265 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991164 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-sysctl-d\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.030265 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991195 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-run\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.030265 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991218 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-lib-modules\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.030265 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991242 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj25s\" (UniqueName: \"kubernetes.io/projected/42a54609-e09f-4ded-a4c8-c35f6b6365c9-kube-api-access-mj25s\") pod \"iptables-alerter-x6wbt\" (UID: \"42a54609-e09f-4ded-a4c8-c35f6b6365c9\") " pod="openshift-network-operator/iptables-alerter-x6wbt" Jul 13 18:58:37.030265 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991266 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-multus-cni-dir\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.030265 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991289 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-log-socket\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.030265 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991312 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/763b955f-d72c-46af-8046-87d4b90b7f12-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:37.030265 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991335 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x544r\" (UniqueName: \"kubernetes.io/projected/763b955f-d72c-46af-8046-87d4b90b7f12-kube-api-access-x544r\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:37.030265 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991341 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-sysctl-d\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.030265 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991357 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/42a54609-e09f-4ded-a4c8-c35f6b6365c9-iptables-alerter-script\") pod \"iptables-alerter-x6wbt\" (UID: \"42a54609-e09f-4ded-a4c8-c35f6b6365c9\") " pod="openshift-network-operator/iptables-alerter-x6wbt" Jul 13 18:58:37.030265 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991391 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-os-release\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.030265 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991401 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-lib-modules\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.030265 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991417 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f079acce-c9cf-4ba0-b003-a176d76fa3ad-agent-certs\") pod \"konnectivity-agent-8v8bj\" (UID: \"f079acce-c9cf-4ba0-b003-a176d76fa3ad\") " pod="kube-system/konnectivity-agent-8v8bj" Jul 13 18:58:37.030265 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991495 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-run\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.030265 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991523 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f079acce-c9cf-4ba0-b003-a176d76fa3ad-konnectivity-ca\") pod \"konnectivity-agent-8v8bj\" (UID: \"f079acce-c9cf-4ba0-b003-a176d76fa3ad\") " pod="kube-system/konnectivity-agent-8v8bj" Jul 13 18:58:37.030265 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991550 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-host-run-multus-certs\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.031391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991576 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-systemd-units\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.031391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991599 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b5a54297-cff5-4224-b5c8-6e1f059d5580-os-release\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:37.031391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991623 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-kubelet-config\") pod \"global-pull-secret-syncer-5ftlh\" (UID: \"e8bde243-4f0c-4fd7-a18c-b5900c1609e8\") " pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:37.031391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991646 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-host\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.031391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991668 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-cnibin\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.031391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991689 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-hostroot\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.031391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991714 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-etc-kubernetes\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.031391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991715 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/763b955f-d72c-46af-8046-87d4b90b7f12-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:37.031391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991744 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5a54297-cff5-4224-b5c8-6e1f059d5580-system-cni-dir\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:37.031391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991741 2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 18:58:37.031391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991772 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-var-lib-openvswitch\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.031391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991798 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-kubelet-config\") pod \"global-pull-secret-syncer-5ftlh\" (UID: \"e8bde243-4f0c-4fd7-a18c-b5900c1609e8\") " pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:37.031391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991819 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnjkj\" (UniqueName: \"kubernetes.io/projected/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-kube-api-access-cnjkj\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.031391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991844 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-sysctl-conf\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.031391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991867 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f28fb900-3954-4e4e-a7a1-527ea570ae72-tmp\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.031391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991892 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-multus-socket-dir-parent\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.031391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991917 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-host-run-netns\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.031391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991931 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/42a54609-e09f-4ded-a4c8-c35f6b6365c9-iptables-alerter-script\") pod \"iptables-alerter-x6wbt\" (UID: \"42a54609-e09f-4ded-a4c8-c35f6b6365c9\") " pod="openshift-network-operator/iptables-alerter-x6wbt" Jul 13 18:58:37.032223 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991942 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-cni-netd\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.032223 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.991990 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.032223 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992014 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-run-systemd\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.032223 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992017 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-multus-cni-dir\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.032223 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992047 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-ovn-node-metrics-cert\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.032223 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992070 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-systemd\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.032223 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992077 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-host\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.032223 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992087 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-var-lib-kubelet\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.032223 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992102 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-host-var-lib-kubelet\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.032223 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992117 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4wz2\" (UniqueName: \"kubernetes.io/projected/1989f7e8-7848-4a80-b6f2-2344cdbe418e-kube-api-access-k4wz2\") pod \"node-ca-xdbcs\" (UID: \"1989f7e8-7848-4a80-b6f2-2344cdbe418e\") " pod="openshift-image-registry/node-ca-xdbcs" Jul 13 18:58:37.032223 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992128 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-cnibin\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.032223 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992131 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-ovnkube-config\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.032223 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992165 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-dbus\") pod \"global-pull-secret-syncer-5ftlh\" (UID: \"e8bde243-4f0c-4fd7-a18c-b5900c1609e8\") " pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:37.032223 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992175 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-var-lib-kubelet\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.032223 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992194 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-sysconfig\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.032223 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992220 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-cni-binary-copy\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.032223 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992238 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-dbus\") pod \"global-pull-secret-syncer-5ftlh\" (UID: \"e8bde243-4f0c-4fd7-a18c-b5900c1609e8\") " pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:37.033211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992248 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1989f7e8-7848-4a80-b6f2-2344cdbe418e-host\") pod \"node-ca-xdbcs\" (UID: \"1989f7e8-7848-4a80-b6f2-2344cdbe418e\") " pod="openshift-image-registry/node-ca-xdbcs" Jul 13 18:58:37.033211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992259 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-hostroot\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.033211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992262 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b5a54297-cff5-4224-b5c8-6e1f059d5580-os-release\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:37.033211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992221 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f079acce-c9cf-4ba0-b003-a176d76fa3ad-konnectivity-ca\") pod \"konnectivity-agent-8v8bj\" (UID: \"f079acce-c9cf-4ba0-b003-a176d76fa3ad\") " pod="kube-system/konnectivity-agent-8v8bj" Jul 13 18:58:37.033211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992298 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-os-release\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.033211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992301 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-multus-socket-dir-parent\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.033211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992321 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-sysconfig\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.033211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992340 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5a54297-cff5-4224-b5c8-6e1f059d5580-system-cni-dir\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:37.033211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992333 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-etc-kubernetes\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.033211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992360 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-host-run-multus-certs\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.033211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992369 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-sysctl-conf\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.033211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992390 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-host-run-netns\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.033211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992398 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-host-var-lib-kubelet\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.033211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992410 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-systemd\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.033211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992415 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b5a54297-cff5-4224-b5c8-6e1f059d5580-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:37.033211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992442 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs\") pod \"network-metrics-daemon-26wx8\" (UID: \"b735dd6a-c571-43a2-a6f7-56a9094519a9\") " pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:37.033211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992480 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/763b955f-d72c-46af-8046-87d4b90b7f12-socket-dir\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:37.033211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992545 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/763b955f-d72c-46af-8046-87d4b90b7f12-device-dir\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:37.033877 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992569 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42a54609-e09f-4ded-a4c8-c35f6b6365c9-host-slash\") pod \"iptables-alerter-x6wbt\" (UID: \"42a54609-e09f-4ded-a4c8-c35f6b6365c9\") " pod="openshift-network-operator/iptables-alerter-x6wbt" Jul 13 18:58:37.033877 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992601 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-host-var-lib-cni-bin\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.033877 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:36.992621 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:37.033877 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992653 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcgv4\" (UniqueName: \"kubernetes.io/projected/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-kube-api-access-zcgv4\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.033877 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992675 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/763b955f-d72c-46af-8046-87d4b90b7f12-socket-dir\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:37.033877 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:36.992687 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs podName:b735dd6a-c571-43a2-a6f7-56a9094519a9 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:37.492668717 +0000 UTC m=+2.969122889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs") pod "network-metrics-daemon-26wx8" (UID: "b735dd6a-c571-43a2-a6f7-56a9094519a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:37.033877 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992691 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-host-var-lib-cni-bin\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.033877 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992570 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-cni-binary-copy\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.033877 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992733 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret\") pod \"global-pull-secret-syncer-5ftlh\" (UID: \"e8bde243-4f0c-4fd7-a18c-b5900c1609e8\") " pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:37.033877 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992811 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/763b955f-d72c-46af-8046-87d4b90b7f12-device-dir\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:37.033877 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.993132 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b5a54297-cff5-4224-b5c8-6e1f059d5580-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:37.033877 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.993189 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-system-cni-dir\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.033877 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.993213 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-host-var-lib-cni-multus\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.033877 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.993261 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nbrd\" (UniqueName: \"kubernetes.io/projected/b5a54297-cff5-4224-b5c8-6e1f059d5580-kube-api-access-2nbrd\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:37.033877 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.993285 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbqzq\" (UniqueName: \"kubernetes.io/projected/32074cd4-753e-4257-9572-b2c21a540c81-kube-api-access-sbqzq\") pod \"node-resolver-jczlv\" (UID: \"32074cd4-753e-4257-9572-b2c21a540c81\") " pod="openshift-dns/node-resolver-jczlv" Jul 13 18:58:37.033877 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.993307 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-run-ovn\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.033877 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.995324 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f079acce-c9cf-4ba0-b003-a176d76fa3ad-agent-certs\") pod \"konnectivity-agent-8v8bj\" (UID: \"f079acce-c9cf-4ba0-b003-a176d76fa3ad\") " pod="kube-system/konnectivity-agent-8v8bj" Jul 13 18:58:37.034905 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.993331 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-modprobe-d\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.034905 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.992842 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42a54609-e09f-4ded-a4c8-c35f6b6365c9-host-slash\") pod \"iptables-alerter-x6wbt\" (UID: \"42a54609-e09f-4ded-a4c8-c35f6b6365c9\") " pod="openshift-network-operator/iptables-alerter-x6wbt" Jul 13 18:58:37.034905 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996249 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-system-cni-dir\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.034905 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996281 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-tuned\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.034905 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996291 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f28fb900-3954-4e4e-a7a1-527ea570ae72-tmp\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.034905 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996306 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5a54297-cff5-4224-b5c8-6e1f059d5580-cni-binary-copy\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:37.034905 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:36.996330 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:37.034905 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996340 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-slash\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.034905 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:36.996390 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret podName:e8bde243-4f0c-4fd7-a18c-b5900c1609e8 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:37.496361379 +0000 UTC m=+2.972815531 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret") pod "global-pull-secret-syncer-5ftlh" (UID: "e8bde243-4f0c-4fd7-a18c-b5900c1609e8") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:37.034905 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996411 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-etc-openvswitch\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.034905 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996439 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-host-var-lib-cni-multus\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.034905 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996471 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-modprobe-d\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.034905 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996478 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.034905 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996515 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-env-overrides\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.034905 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996560 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5a54297-cff5-4224-b5c8-6e1f059d5580-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:37.034905 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996590 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5a54297-cff5-4224-b5c8-6e1f059d5580-cnibin\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:37.034905 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996613 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvrbq\" (UniqueName: \"kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq\") pod \"network-check-target-f4ffp\" (UID: \"b78c80c9-b3b8-40a6-a842-da04dbeddf03\") " pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:37.035652 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996644 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-cni-bin\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.035652 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996667 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-ovnkube-script-lib\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.035652 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996692 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/763b955f-d72c-46af-8046-87d4b90b7f12-sys-fs\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:37.035652 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996712 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5a54297-cff5-4224-b5c8-6e1f059d5580-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:37.035652 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996714 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-kubernetes\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.035652 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996745 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-host-run-k8s-cni-cncf-io\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.035652 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996760 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-kubernetes\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.035652 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996770 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-run-openvswitch\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.035652 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996815 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-node-log\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.035652 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996844 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/763b955f-d72c-46af-8046-87d4b90b7f12-registration-dir\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:37.035652 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996868 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-662nw\" (UniqueName: \"kubernetes.io/projected/f28fb900-3954-4e4e-a7a1-527ea570ae72-kube-api-access-662nw\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.035652 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996925 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5a54297-cff5-4224-b5c8-6e1f059d5580-cnibin\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:37.035652 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996973 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/763b955f-d72c-46af-8046-87d4b90b7f12-sys-fs\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:37.035652 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.996998 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/32074cd4-753e-4257-9572-b2c21a540c81-hosts-file\") pod \"node-resolver-jczlv\" (UID: \"32074cd4-753e-4257-9572-b2c21a540c81\") " pod="openshift-dns/node-resolver-jczlv" Jul 13 18:58:37.035652 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.997023 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpd78\" (UniqueName: \"kubernetes.io/projected/b735dd6a-c571-43a2-a6f7-56a9094519a9-kube-api-access-qpd78\") pod \"network-metrics-daemon-26wx8\" (UID: \"b735dd6a-c571-43a2-a6f7-56a9094519a9\") " pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:37.035652 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.997031 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5a54297-cff5-4224-b5c8-6e1f059d5580-cni-binary-copy\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:37.035652 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.997047 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-multus-daemon-config\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.036596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.997070 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1989f7e8-7848-4a80-b6f2-2344cdbe418e-serviceca\") pod \"node-ca-xdbcs\" (UID: \"1989f7e8-7848-4a80-b6f2-2344cdbe418e\") " pod="openshift-image-registry/node-ca-xdbcs" Jul 13 18:58:37.036596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.997094 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/763b955f-d72c-46af-8046-87d4b90b7f12-etc-selinux\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:37.036596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.997097 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-host-run-k8s-cni-cncf-io\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.036596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.997116 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-sys\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.036596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.997321 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-multus-conf-dir\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.036596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.997340 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/763b955f-d72c-46af-8046-87d4b90b7f12-registration-dir\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:37.036596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.997357 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b5a54297-cff5-4224-b5c8-6e1f059d5580-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:37.036596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.997617 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f28fb900-3954-4e4e-a7a1-527ea570ae72-sys\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.036596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.997713 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32074cd4-753e-4257-9572-b2c21a540c81-tmp-dir\") pod \"node-resolver-jczlv\" (UID: \"32074cd4-753e-4257-9572-b2c21a540c81\") " pod="openshift-dns/node-resolver-jczlv" Jul 13 18:58:37.036596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.997744 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-kubelet\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.036596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.997750 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-multus-daemon-config\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.036596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.997816 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-multus-conf-dir\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.036596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.997902 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/763b955f-d72c-46af-8046-87d4b90b7f12-etc-selinux\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:37.036596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.998324 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b5a54297-cff5-4224-b5c8-6e1f059d5580-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:37.036596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.998756 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f28fb900-3954-4e4e-a7a1-527ea570ae72-etc-tuned\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.036596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.999374 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj25s\" (UniqueName: \"kubernetes.io/projected/42a54609-e09f-4ded-a4c8-c35f6b6365c9-kube-api-access-mj25s\") pod \"iptables-alerter-x6wbt\" (UID: \"42a54609-e09f-4ded-a4c8-c35f6b6365c9\") " pod="openshift-network-operator/iptables-alerter-x6wbt" Jul 13 18:58:37.036596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.999622 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnjkj\" (UniqueName: \"kubernetes.io/projected/1f5cf377-1e0d-49e3-ba52-dee61f029c7f-kube-api-access-cnjkj\") pod \"multus-4pbnd\" (UID: \"1f5cf377-1e0d-49e3-ba52-dee61f029c7f\") " pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.037355 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:36.999984 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x544r\" (UniqueName: \"kubernetes.io/projected/763b955f-d72c-46af-8046-87d4b90b7f12-kube-api-access-x544r\") pod \"aws-ebs-csi-driver-node-xmsqw\" (UID: \"763b955f-d72c-46af-8046-87d4b90b7f12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:37.037355 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.005621 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nbrd\" (UniqueName: \"kubernetes.io/projected/b5a54297-cff5-4224-b5c8-6e1f059d5580-kube-api-access-2nbrd\") pod \"multus-additional-cni-plugins-qgq48\" (UID: \"b5a54297-cff5-4224-b5c8-6e1f059d5580\") " pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:37.037355 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.006578 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-662nw\" (UniqueName: \"kubernetes.io/projected/f28fb900-3954-4e4e-a7a1-527ea570ae72-kube-api-access-662nw\") pod \"tuned-slbqz\" (UID: \"f28fb900-3954-4e4e-a7a1-527ea570ae72\") " pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.037355 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.007651 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpd78\" (UniqueName: \"kubernetes.io/projected/b735dd6a-c571-43a2-a6f7-56a9094519a9-kube-api-access-qpd78\") pod \"network-metrics-daemon-26wx8\" (UID: \"b735dd6a-c571-43a2-a6f7-56a9094519a9\") " pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:37.118678 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.052141 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal" event={"ID":"9f50767bdb9b137e0ddb50e76b305359","Type":"ContainerStarted","Data":"b5253404c2d273e34b940907b3db94c1bfc3f717a902e76ddb6c53439a35b45a"} Jul 13 18:58:37.118678 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.053253 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" event={"ID":"2a9da3169af3e3a8e9cc39dc3e8dbe13","Type":"ContainerStarted","Data":"755abd8ef01f04c471a9c4afecb97f727235160959dc4144db92782359d8b82c"} Jul 13 18:58:37.118678 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098209 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvrbq\" (UniqueName: \"kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq\") pod \"network-check-target-f4ffp\" (UID: \"b78c80c9-b3b8-40a6-a842-da04dbeddf03\") " pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:37.118678 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098250 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-cni-bin\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.118678 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098273 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-ovnkube-script-lib\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.118678 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098297 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-run-openvswitch\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.118678 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098339 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-node-log\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.118678 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098363 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/32074cd4-753e-4257-9572-b2c21a540c81-hosts-file\") pod \"node-resolver-jczlv\" (UID: \"32074cd4-753e-4257-9572-b2c21a540c81\") " pod="openshift-dns/node-resolver-jczlv" Jul 13 18:58:37.118678 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098391 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-run-openvswitch\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.118678 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098387 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1989f7e8-7848-4a80-b6f2-2344cdbe418e-serviceca\") pod \"node-ca-xdbcs\" (UID: \"1989f7e8-7848-4a80-b6f2-2344cdbe418e\") " pod="openshift-image-registry/node-ca-xdbcs" Jul 13 18:58:37.118678 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098447 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32074cd4-753e-4257-9572-b2c21a540c81-tmp-dir\") pod \"node-resolver-jczlv\" (UID: \"32074cd4-753e-4257-9572-b2c21a540c81\") " pod="openshift-dns/node-resolver-jczlv" Jul 13 18:58:37.118678 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098469 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-kubelet\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.118678 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098357 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-cni-bin\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.118678 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098491 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-run-netns\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.118678 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098515 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/32074cd4-753e-4257-9572-b2c21a540c81-hosts-file\") pod \"node-resolver-jczlv\" (UID: \"32074cd4-753e-4257-9572-b2c21a540c81\") " pod="openshift-dns/node-resolver-jczlv" Jul 13 18:58:37.118678 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098537 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-run-netns\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.118678 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098545 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-kubelet\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.118678 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098550 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-log-socket\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.120958 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098544 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-node-log\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.120958 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098594 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-log-socket\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.120958 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098735 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32074cd4-753e-4257-9572-b2c21a540c81-tmp-dir\") pod \"node-resolver-jczlv\" (UID: \"32074cd4-753e-4257-9572-b2c21a540c81\") " pod="openshift-dns/node-resolver-jczlv" Jul 13 18:58:37.120958 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098725 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-systemd-units\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.120958 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098813 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-systemd-units\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.120958 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098832 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-var-lib-openvswitch\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.120958 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098860 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1989f7e8-7848-4a80-b6f2-2344cdbe418e-serviceca\") pod \"node-ca-xdbcs\" (UID: \"1989f7e8-7848-4a80-b6f2-2344cdbe418e\") " pod="openshift-image-registry/node-ca-xdbcs" Jul 13 18:58:37.120958 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098884 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-cni-netd\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.120958 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098919 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-var-lib-openvswitch\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.120958 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098926 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.120958 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098957 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-cni-netd\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.120958 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.098974 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-run-systemd\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.120958 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.099053 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.120958 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.099054 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-ovnkube-script-lib\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.120958 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.099065 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-ovn-node-metrics-cert\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.120958 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.099089 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-run-systemd\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.120958 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.099115 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4wz2\" (UniqueName: \"kubernetes.io/projected/1989f7e8-7848-4a80-b6f2-2344cdbe418e-kube-api-access-k4wz2\") pod \"node-ca-xdbcs\" (UID: \"1989f7e8-7848-4a80-b6f2-2344cdbe418e\") " pod="openshift-image-registry/node-ca-xdbcs" Jul 13 18:58:37.121744 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.099140 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-ovnkube-config\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.121744 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.099183 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1989f7e8-7848-4a80-b6f2-2344cdbe418e-host\") pod \"node-ca-xdbcs\" (UID: \"1989f7e8-7848-4a80-b6f2-2344cdbe418e\") " pod="openshift-image-registry/node-ca-xdbcs" Jul 13 18:58:37.121744 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.099631 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcgv4\" (UniqueName: \"kubernetes.io/projected/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-kube-api-access-zcgv4\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.121744 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.099759 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbqzq\" (UniqueName: \"kubernetes.io/projected/32074cd4-753e-4257-9572-b2c21a540c81-kube-api-access-sbqzq\") pod \"node-resolver-jczlv\" (UID: \"32074cd4-753e-4257-9572-b2c21a540c81\") " pod="openshift-dns/node-resolver-jczlv" Jul 13 18:58:37.121744 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.099802 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-run-ovn\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.121744 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.099831 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1989f7e8-7848-4a80-b6f2-2344cdbe418e-host\") pod \"node-ca-xdbcs\" (UID: \"1989f7e8-7848-4a80-b6f2-2344cdbe418e\") " pod="openshift-image-registry/node-ca-xdbcs" Jul 13 18:58:37.121744 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.099859 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-run-ovn\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.121744 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.100025 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-slash\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.121744 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.100062 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-etc-openvswitch\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.121744 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.100123 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-slash\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.121744 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.100239 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-etc-openvswitch\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.121744 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.100263 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-ovnkube-config\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.121744 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.100358 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.121744 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.102312 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.121744 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.102352 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-env-overrides\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.121744 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.102622 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-ovn-node-metrics-cert\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.121744 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.102807 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-env-overrides\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.121744 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:37.109994 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:58:37.122494 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:37.110014 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:58:37.122494 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:37.110027 2574 projected.go:196] Error preparing data for projected volume kube-api-access-vvrbq for pod openshift-network-diagnostics/network-check-target-f4ffp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:37.122494 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:37.110103 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq podName:b78c80c9-b3b8-40a6-a842-da04dbeddf03 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:37.610086422 +0000 UTC m=+3.086540592 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vvrbq" (UniqueName: "kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq") pod "network-check-target-f4ffp" (UID: "b78c80c9-b3b8-40a6-a842-da04dbeddf03") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:37.122494 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.112370 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4wz2\" (UniqueName: \"kubernetes.io/projected/1989f7e8-7848-4a80-b6f2-2344cdbe418e-kube-api-access-k4wz2\") pod \"node-ca-xdbcs\" (UID: \"1989f7e8-7848-4a80-b6f2-2344cdbe418e\") " pod="openshift-image-registry/node-ca-xdbcs" Jul 13 18:58:37.122494 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.112631 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcgv4\" (UniqueName: \"kubernetes.io/projected/6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7-kube-api-access-zcgv4\") pod \"ovnkube-node-79lb6\" (UID: \"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.122494 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.112695 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbqzq\" (UniqueName: \"kubernetes.io/projected/32074cd4-753e-4257-9572-b2c21a540c81-kube-api-access-sbqzq\") pod \"node-resolver-jczlv\" (UID: \"32074cd4-753e-4257-9572-b2c21a540c81\") " pod="openshift-dns/node-resolver-jczlv" Jul 13 18:58:37.219270 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.196922 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x6wbt" Jul 13 18:58:37.219270 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.204916 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" Jul 13 18:58:37.219270 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:37.206258 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a54609_e09f_4ded_a4c8_c35f6b6365c9.slice/crio-bf60aa747b886a1957b0afa177e9aa905fa74122028325e559615f35005deb7b WatchSource:0}: Error finding container bf60aa747b886a1957b0afa177e9aa905fa74122028325e559615f35005deb7b: Status 404 returned error can't find the container with id bf60aa747b886a1957b0afa177e9aa905fa74122028325e559615f35005deb7b Jul 13 18:58:37.219270 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.211551 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-slbqz" Jul 13 18:58:37.219270 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:37.215553 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod763b955f_d72c_46af_8046_87d4b90b7f12.slice/crio-8c922ce45219e2e3acfaf91a70b85d63193c8df4786c814234750d2458012703 WatchSource:0}: Error finding container 8c922ce45219e2e3acfaf91a70b85d63193c8df4786c814234750d2458012703: Status 404 returned error can't find the container with id 8c922ce45219e2e3acfaf91a70b85d63193c8df4786c814234750d2458012703 Jul 13 18:58:37.219270 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.218959 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4pbnd" Jul 13 18:58:37.223828 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:37.221211 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf28fb900_3954_4e4e_a7a1_527ea570ae72.slice/crio-be73c23a90535c205ce9be92109ef47393658860028a80a19b0df03f9d52f9d2 WatchSource:0}: Error finding container be73c23a90535c205ce9be92109ef47393658860028a80a19b0df03f9d52f9d2: Status 404 returned error can't find the container with id be73c23a90535c205ce9be92109ef47393658860028a80a19b0df03f9d52f9d2 Jul 13 18:58:37.284835 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.226217 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qgq48" Jul 13 18:58:37.284835 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.233307 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8v8bj" Jul 13 18:58:37.284835 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.243841 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jczlv" Jul 13 18:58:37.284835 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.253226 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xdbcs" Jul 13 18:58:37.284835 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.258834 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:58:37.299440 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:37.299419 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f5cf377_1e0d_49e3_ba52_dee61f029c7f.slice/crio-5b4e635d1dcdd04c1afae6d1fa809b652a8330d5a6d300f4ed269f8f403d3c97 WatchSource:0}: Error finding container 5b4e635d1dcdd04c1afae6d1fa809b652a8330d5a6d300f4ed269f8f403d3c97: Status 404 returned error can't find the container with id 5b4e635d1dcdd04c1afae6d1fa809b652a8330d5a6d300f4ed269f8f403d3c97 Jul 13 18:58:37.300056 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:37.300010 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a54297_cff5_4224_b5c8_6e1f059d5580.slice/crio-f79f68b52ad7236a183b2d0aef58b51e2d6471e5f8dbd134f3354f8be7d9f9bf WatchSource:0}: Error finding container f79f68b52ad7236a183b2d0aef58b51e2d6471e5f8dbd134f3354f8be7d9f9bf: Status 404 returned error can't find the container with id f79f68b52ad7236a183b2d0aef58b51e2d6471e5f8dbd134f3354f8be7d9f9bf Jul 13 18:58:37.309190 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:37.309163 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f2dadfd_ce72_47b8_ac79_ca34bfbc27e7.slice/crio-fcdffdff255b2da893f648a2802d2d5dd6c69341b0685a024d6617e058ec292d WatchSource:0}: Error finding container fcdffdff255b2da893f648a2802d2d5dd6c69341b0685a024d6617e058ec292d: Status 404 returned error can't find the container with id fcdffdff255b2da893f648a2802d2d5dd6c69341b0685a024d6617e058ec292d Jul 13 18:58:37.311183 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:37.311082 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf079acce_c9cf_4ba0_b003_a176d76fa3ad.slice/crio-b529ddf3087198ad9f8cf861f9e02fd7aafd494f16eed043a16064cb81dc26cc WatchSource:0}: Error finding container b529ddf3087198ad9f8cf861f9e02fd7aafd494f16eed043a16064cb81dc26cc: Status 404 returned error can't find the container with id b529ddf3087198ad9f8cf861f9e02fd7aafd494f16eed043a16064cb81dc26cc Jul 13 18:58:37.312507 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:37.312485 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1989f7e8_7848_4a80_b6f2_2344cdbe418e.slice/crio-ac8f07e6bf413d463cf24b39a4f11527a93438c1b2cc589c636d5ffe6b7ea575 WatchSource:0}: Error finding container ac8f07e6bf413d463cf24b39a4f11527a93438c1b2cc589c636d5ffe6b7ea575: Status 404 returned error can't find the container with id ac8f07e6bf413d463cf24b39a4f11527a93438c1b2cc589c636d5ffe6b7ea575 Jul 13 18:58:37.506103 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.506017 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs\") pod \"network-metrics-daemon-26wx8\" (UID: \"b735dd6a-c571-43a2-a6f7-56a9094519a9\") " pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:37.506103 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.506066 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret\") pod \"global-pull-secret-syncer-5ftlh\" (UID: \"e8bde243-4f0c-4fd7-a18c-b5900c1609e8\") " pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:37.506300 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:37.506169 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:37.506300 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:37.506169 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:37.506300 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:37.506225 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret podName:e8bde243-4f0c-4fd7-a18c-b5900c1609e8 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:38.506208917 +0000 UTC m=+3.982663075 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret") pod "global-pull-secret-syncer-5ftlh" (UID: "e8bde243-4f0c-4fd7-a18c-b5900c1609e8") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:37.506300 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:37.506278 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs podName:b735dd6a-c571-43a2-a6f7-56a9094519a9 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:38.506259735 +0000 UTC m=+3.982713899 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs") pod "network-metrics-daemon-26wx8" (UID: "b735dd6a-c571-43a2-a6f7-56a9094519a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:37.708688 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.708660 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvrbq\" (UniqueName: \"kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq\") pod \"network-check-target-f4ffp\" (UID: \"b78c80c9-b3b8-40a6-a842-da04dbeddf03\") " pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:37.708863 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:37.708815 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:58:37.708863 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:37.708849 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:58:37.708863 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:37.708859 2574 projected.go:196] Error preparing data for projected volume kube-api-access-vvrbq for pod openshift-network-diagnostics/network-check-target-f4ffp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:37.708972 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:37.708911 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq podName:b78c80c9-b3b8-40a6-a842-da04dbeddf03 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:38.70889546 +0000 UTC m=+4.185349614 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vvrbq" (UniqueName: "kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq") pod "network-check-target-f4ffp" (UID: "b78c80c9-b3b8-40a6-a842-da04dbeddf03") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:37.926633 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.926508 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 18:53:36 +0000 UTC" deadline="2027-12-10 09:10:55.762804766 +0000 UTC" Jul 13 18:58:37.926633 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:37.926567 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12350h12m17.836241804s" Jul 13 18:58:38.049086 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:38.049057 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:38.049229 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:38.049177 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:58:38.056918 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:38.056884 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" event={"ID":"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7","Type":"ContainerStarted","Data":"fcdffdff255b2da893f648a2802d2d5dd6c69341b0685a024d6617e058ec292d"} Jul 13 18:58:38.058988 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:38.058959 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-slbqz" event={"ID":"f28fb900-3954-4e4e-a7a1-527ea570ae72","Type":"ContainerStarted","Data":"be73c23a90535c205ce9be92109ef47393658860028a80a19b0df03f9d52f9d2"} Jul 13 18:58:38.060318 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:38.060256 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" event={"ID":"763b955f-d72c-46af-8046-87d4b90b7f12","Type":"ContainerStarted","Data":"8c922ce45219e2e3acfaf91a70b85d63193c8df4786c814234750d2458012703"} Jul 13 18:58:38.063148 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:38.063119 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x6wbt" event={"ID":"42a54609-e09f-4ded-a4c8-c35f6b6365c9","Type":"ContainerStarted","Data":"bf60aa747b886a1957b0afa177e9aa905fa74122028325e559615f35005deb7b"} Jul 13 18:58:38.064248 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:38.064225 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xdbcs" event={"ID":"1989f7e8-7848-4a80-b6f2-2344cdbe418e","Type":"ContainerStarted","Data":"ac8f07e6bf413d463cf24b39a4f11527a93438c1b2cc589c636d5ffe6b7ea575"} Jul 13 18:58:38.065371 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:38.065350 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8v8bj" event={"ID":"f079acce-c9cf-4ba0-b003-a176d76fa3ad","Type":"ContainerStarted","Data":"b529ddf3087198ad9f8cf861f9e02fd7aafd494f16eed043a16064cb81dc26cc"} Jul 13 18:58:38.066942 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:38.066887 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qgq48" event={"ID":"b5a54297-cff5-4224-b5c8-6e1f059d5580","Type":"ContainerStarted","Data":"f79f68b52ad7236a183b2d0aef58b51e2d6471e5f8dbd134f3354f8be7d9f9bf"} Jul 13 18:58:38.068749 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:38.068722 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4pbnd" event={"ID":"1f5cf377-1e0d-49e3-ba52-dee61f029c7f","Type":"ContainerStarted","Data":"5b4e635d1dcdd04c1afae6d1fa809b652a8330d5a6d300f4ed269f8f403d3c97"} Jul 13 18:58:38.363128 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:58:38.363091 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32074cd4_753e_4257_9572_b2c21a540c81.slice/crio-e9c965a71a6fbe989e85b5e967a14c50075988f5039254784defc44bd1743acf WatchSource:0}: Error finding container e9c965a71a6fbe989e85b5e967a14c50075988f5039254784defc44bd1743acf: Status 404 returned error can't find the container with id e9c965a71a6fbe989e85b5e967a14c50075988f5039254784defc44bd1743acf Jul 13 18:58:38.513332 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:38.513305 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs\") pod \"network-metrics-daemon-26wx8\" (UID: \"b735dd6a-c571-43a2-a6f7-56a9094519a9\") " pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:38.513493 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:38.513350 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret\") pod \"global-pull-secret-syncer-5ftlh\" (UID: \"e8bde243-4f0c-4fd7-a18c-b5900c1609e8\") " pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:38.513493 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:38.513463 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:38.513594 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:38.513508 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:38.513594 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:38.513528 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs podName:b735dd6a-c571-43a2-a6f7-56a9094519a9 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:40.513509299 +0000 UTC m=+5.989963469 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs") pod "network-metrics-daemon-26wx8" (UID: "b735dd6a-c571-43a2-a6f7-56a9094519a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:38.513594 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:38.513551 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret podName:e8bde243-4f0c-4fd7-a18c-b5900c1609e8 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:40.51353723 +0000 UTC m=+5.989991387 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret") pod "global-pull-secret-syncer-5ftlh" (UID: "e8bde243-4f0c-4fd7-a18c-b5900c1609e8") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:38.716865 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:38.715951 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvrbq\" (UniqueName: \"kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq\") pod \"network-check-target-f4ffp\" (UID: \"b78c80c9-b3b8-40a6-a842-da04dbeddf03\") " pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:38.716865 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:38.716358 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:58:38.716865 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:38.716376 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:58:38.716865 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:38.716389 2574 projected.go:196] Error preparing data for projected volume kube-api-access-vvrbq for pod openshift-network-diagnostics/network-check-target-f4ffp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:38.716865 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:38.716440 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq podName:b78c80c9-b3b8-40a6-a842-da04dbeddf03 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:40.716425213 +0000 UTC m=+6.192879366 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vvrbq" (UniqueName: "kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq") pod "network-check-target-f4ffp" (UID: "b78c80c9-b3b8-40a6-a842-da04dbeddf03") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:38.927718 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:38.927635 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 18:53:36 +0000 UTC" deadline="2027-12-06 23:21:06.282174236 +0000 UTC" Jul 13 18:58:38.927718 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:38.927688 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12268h22m27.354491645s" Jul 13 18:58:39.051038 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:39.050973 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:39.051175 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:39.051093 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:58:39.051482 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:39.051464 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:39.051566 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:39.051549 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:58:39.074484 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:39.074451 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jczlv" event={"ID":"32074cd4-753e-4257-9572-b2c21a540c81","Type":"ContainerStarted","Data":"e9c965a71a6fbe989e85b5e967a14c50075988f5039254784defc44bd1743acf"} Jul 13 18:58:39.085261 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:39.085190 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal" event={"ID":"9f50767bdb9b137e0ddb50e76b305359","Type":"ContainerStarted","Data":"9edb1f7f03cdf54181b6349890d3353136573eec52f258674b2a2b9e6694dfff"} Jul 13 18:58:39.089404 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:39.089369 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" event={"ID":"2a9da3169af3e3a8e9cc39dc3e8dbe13","Type":"ContainerStarted","Data":"2c148f65a814b412070119e24133cbcd2ea836376e66080561795ac071af7f98"} Jul 13 18:58:39.115109 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:39.115056 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal" podStartSLOduration=3.115036546 podStartE2EDuration="3.115036546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:39.100880862 +0000 UTC m=+4.577335033" watchObservedRunningTime="2026-07-13 18:58:39.115036546 +0000 UTC m=+4.591490718" Jul 13 18:58:40.049274 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:40.048684 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:40.049274 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:40.048861 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:58:40.104983 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:40.104173 2574 generic.go:356] "Generic (PLEG): container finished" podID="2a9da3169af3e3a8e9cc39dc3e8dbe13" containerID="2c148f65a814b412070119e24133cbcd2ea836376e66080561795ac071af7f98" exitCode=0 Jul 13 18:58:40.104983 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:40.104847 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" event={"ID":"2a9da3169af3e3a8e9cc39dc3e8dbe13","Type":"ContainerDied","Data":"2c148f65a814b412070119e24133cbcd2ea836376e66080561795ac071af7f98"} Jul 13 18:58:40.534476 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:40.534362 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs\") pod \"network-metrics-daemon-26wx8\" (UID: \"b735dd6a-c571-43a2-a6f7-56a9094519a9\") " pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:40.534476 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:40.534423 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret\") pod \"global-pull-secret-syncer-5ftlh\" (UID: \"e8bde243-4f0c-4fd7-a18c-b5900c1609e8\") " pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:40.534695 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:40.534581 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:40.534695 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:40.534636 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret podName:e8bde243-4f0c-4fd7-a18c-b5900c1609e8 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:44.534619041 +0000 UTC m=+10.011073213 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret") pod "global-pull-secret-syncer-5ftlh" (UID: "e8bde243-4f0c-4fd7-a18c-b5900c1609e8") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:40.535074 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:40.535057 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:40.535175 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:40.535100 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs podName:b735dd6a-c571-43a2-a6f7-56a9094519a9 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:44.535088522 +0000 UTC m=+10.011542674 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs") pod "network-metrics-daemon-26wx8" (UID: "b735dd6a-c571-43a2-a6f7-56a9094519a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:40.736602 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:40.736547 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvrbq\" (UniqueName: \"kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq\") pod \"network-check-target-f4ffp\" (UID: \"b78c80c9-b3b8-40a6-a842-da04dbeddf03\") " pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:40.736774 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:40.736756 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:58:40.736876 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:40.736777 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:58:40.736876 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:40.736809 2574 projected.go:196] Error preparing data for projected volume kube-api-access-vvrbq for pod openshift-network-diagnostics/network-check-target-f4ffp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:40.736876 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:40.736870 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq podName:b78c80c9-b3b8-40a6-a842-da04dbeddf03 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:44.736849809 +0000 UTC m=+10.213303965 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vvrbq" (UniqueName: "kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq") pod "network-check-target-f4ffp" (UID: "b78c80c9-b3b8-40a6-a842-da04dbeddf03") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:41.049874 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:41.049843 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:41.050337 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:41.049968 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:58:41.050337 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:41.050108 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:41.050337 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:41.050210 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:58:42.048692 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:42.048659 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:42.048892 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:42.048829 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:58:43.048516 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:43.048478 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:43.048954 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:43.048614 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:58:43.048954 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:43.048662 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:43.048954 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:43.048754 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:58:44.049102 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:44.049041 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:44.049643 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:44.049231 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:58:44.567922 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:44.567883 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs\") pod \"network-metrics-daemon-26wx8\" (UID: \"b735dd6a-c571-43a2-a6f7-56a9094519a9\") " pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:44.568137 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:44.567945 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret\") pod \"global-pull-secret-syncer-5ftlh\" (UID: \"e8bde243-4f0c-4fd7-a18c-b5900c1609e8\") " pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:44.568137 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:44.568054 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:44.568137 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:44.568077 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:44.568137 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:44.568135 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs podName:b735dd6a-c571-43a2-a6f7-56a9094519a9 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:52.568112548 +0000 UTC m=+18.044566716 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs") pod "network-metrics-daemon-26wx8" (UID: "b735dd6a-c571-43a2-a6f7-56a9094519a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:44.568390 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:44.568157 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret podName:e8bde243-4f0c-4fd7-a18c-b5900c1609e8 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:52.568146 +0000 UTC m=+18.044600152 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret") pod "global-pull-secret-syncer-5ftlh" (UID: "e8bde243-4f0c-4fd7-a18c-b5900c1609e8") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:44.770005 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:44.769939 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvrbq\" (UniqueName: \"kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq\") pod \"network-check-target-f4ffp\" (UID: \"b78c80c9-b3b8-40a6-a842-da04dbeddf03\") " pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:44.770190 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:44.770107 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:58:44.770190 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:44.770133 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:58:44.770190 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:44.770147 2574 projected.go:196] Error preparing data for projected volume kube-api-access-vvrbq for pod openshift-network-diagnostics/network-check-target-f4ffp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:44.770353 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:44.770214 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq podName:b78c80c9-b3b8-40a6-a842-da04dbeddf03 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:52.770193816 +0000 UTC m=+18.246647980 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vvrbq" (UniqueName: "kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq") pod "network-check-target-f4ffp" (UID: "b78c80c9-b3b8-40a6-a842-da04dbeddf03") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:45.049949 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:45.049485 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:45.049949 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:45.049592 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:58:45.049949 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:45.049797 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:45.049949 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:45.049936 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:58:46.049315 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:46.048712 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:46.049315 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:46.048878 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:58:47.048668 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:47.048569 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:47.049271 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:47.049239 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:58:47.052809 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:47.050838 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:47.052809 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:47.050983 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:58:48.048324 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:48.048229 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:48.048500 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:48.048382 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:58:49.049157 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:49.049123 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:49.050037 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:49.049254 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:58:49.050037 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:49.049298 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:49.050037 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:49.049395 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:58:50.048988 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:50.048954 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:50.049158 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:50.049088 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:58:51.049145 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:51.049096 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:51.049600 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:51.049270 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:58:51.049600 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:51.049310 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:51.049600 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:51.049423 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:58:52.048840 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:52.048806 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:52.049041 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:52.048941 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:58:52.632723 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:52.632687 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs\") pod \"network-metrics-daemon-26wx8\" (UID: \"b735dd6a-c571-43a2-a6f7-56a9094519a9\") " pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:52.633163 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:52.632735 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret\") pod \"global-pull-secret-syncer-5ftlh\" (UID: \"e8bde243-4f0c-4fd7-a18c-b5900c1609e8\") " pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:52.633163 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:52.632857 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:52.633163 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:52.632863 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:52.633163 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:52.632928 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs podName:b735dd6a-c571-43a2-a6f7-56a9094519a9 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:08.632908132 +0000 UTC m=+34.109362301 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs") pod "network-metrics-daemon-26wx8" (UID: "b735dd6a-c571-43a2-a6f7-56a9094519a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:52.633163 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:52.632951 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret podName:e8bde243-4f0c-4fd7-a18c-b5900c1609e8 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:08.632940685 +0000 UTC m=+34.109394837 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret") pod "global-pull-secret-syncer-5ftlh" (UID: "e8bde243-4f0c-4fd7-a18c-b5900c1609e8") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:52.833836 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:52.833779 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvrbq\" (UniqueName: \"kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq\") pod \"network-check-target-f4ffp\" (UID: \"b78c80c9-b3b8-40a6-a842-da04dbeddf03\") " pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:52.833973 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:52.833945 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:58:52.834010 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:52.833979 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:58:52.834010 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:52.833994 2574 projected.go:196] Error preparing data for projected volume kube-api-access-vvrbq for pod openshift-network-diagnostics/network-check-target-f4ffp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:52.834122 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:52.834067 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq podName:b78c80c9-b3b8-40a6-a842-da04dbeddf03 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:08.83404577 +0000 UTC m=+34.310499937 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vvrbq" (UniqueName: "kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq") pod "network-check-target-f4ffp" (UID: "b78c80c9-b3b8-40a6-a842-da04dbeddf03") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:53.049239 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:53.049157 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:53.049239 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:53.049202 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:53.049436 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:53.049302 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:58:53.049502 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:53.049432 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:58:54.048645 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:54.048613 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:54.049099 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:54.048738 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:58:55.049848 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:55.049815 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:55.050311 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:55.049935 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:58:55.050311 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:55.049990 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:55.050311 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:55.050096 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:58:56.048510 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:56.048481 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:56.048626 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:56.048606 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:58:56.139971 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:56.139861 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" event={"ID":"2a9da3169af3e3a8e9cc39dc3e8dbe13","Type":"ContainerStarted","Data":"18baece3d790fb9afb95688d3d7200ad7da1873c02d5c1a147762c1fb7caa286"} Jul 13 18:58:56.142210 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:56.142128 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-slbqz" event={"ID":"f28fb900-3954-4e4e-a7a1-527ea570ae72","Type":"ContainerStarted","Data":"201680f9fb3e7cc2d7f1a952b395dbd9a70790e2970c75b0943226fd3c856e05"} Jul 13 18:58:56.143710 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:56.143681 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" event={"ID":"763b955f-d72c-46af-8046-87d4b90b7f12","Type":"ContainerStarted","Data":"f2748b16ba4ebc4b97fac2b847630f000c0b8e55b4b8c39dafd2bb1ef3e327f1"} Jul 13 18:58:56.145294 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:56.145221 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8v8bj" event={"ID":"f079acce-c9cf-4ba0-b003-a176d76fa3ad","Type":"ContainerStarted","Data":"b9a530f97111aeb7db938cff56bd356d69bee2e42c25c9830d46e399c04b7bdc"} Jul 13 18:58:56.185695 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:56.185477 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" podStartSLOduration=20.185460194 podStartE2EDuration="20.185460194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:56.171179143 +0000 UTC m=+21.647633318" watchObservedRunningTime="2026-07-13 18:58:56.185460194 +0000 UTC m=+21.661914367" Jul 13 18:58:56.185939 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:56.185893 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8v8bj" podStartSLOduration=11.895339537 podStartE2EDuration="22.185879304s" totalImagesPullingTime="10.290539767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:56.185159992 +0000 UTC m=+21.661614188" watchObservedRunningTime="2026-07-13 18:58:56.185879304 +0000 UTC m=+21.662333479" Jul 13 18:58:56.202020 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:56.201979 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-slbqz" podStartSLOduration=3.489530877 podStartE2EDuration="22.201969868s" totalImagesPullingTime="18.712438991s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:56.201766494 +0000 UTC m=+21.678220669" watchObservedRunningTime="2026-07-13 18:58:56.201969868 +0000 UTC m=+21.678424111" Jul 13 18:58:56.419658 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:56.419628 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:58:56.498522 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:56.498481 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:58:57.049359 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.049286 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:57.049359 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.049320 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:57.049480 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:57.049415 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:58:57.049525 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:57.049493 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:58:57.148299 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.148264 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jczlv" event={"ID":"32074cd4-753e-4257-9572-b2c21a540c81","Type":"ContainerStarted","Data":"c09516d20a1526775267c2449de65ceff40e73bd09971e3dc56d07880354ea57"} Jul 13 18:58:57.149634 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.149613 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xdbcs" event={"ID":"1989f7e8-7848-4a80-b6f2-2344cdbe418e","Type":"ContainerStarted","Data":"754c821ce8c8108a043eb07a4c00f2e28d915ac950bb94944c9f55d0e89c9dd4"} Jul 13 18:58:57.151014 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.150989 2574 generic.go:356] "Generic (PLEG): container finished" podID="b5a54297-cff5-4224-b5c8-6e1f059d5580" containerID="9e38b58126d461fd4b8af5169fe919ba594b0954f1e97e097640cefe1ab0bd83" exitCode=0 Jul 13 18:58:57.151115 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.151071 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qgq48" event={"ID":"b5a54297-cff5-4224-b5c8-6e1f059d5580","Type":"ContainerDied","Data":"9e38b58126d461fd4b8af5169fe919ba594b0954f1e97e097640cefe1ab0bd83"} Jul 13 18:58:57.152453 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.152429 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4pbnd" event={"ID":"1f5cf377-1e0d-49e3-ba52-dee61f029c7f","Type":"ContainerStarted","Data":"a471a4cc689c5cb4a36e8a4c39647582622955f74008eb3c94157cdf96d59470"} Jul 13 18:58:57.153890 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.153867 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x6wbt" event={"ID":"42a54609-e09f-4ded-a4c8-c35f6b6365c9","Type":"ContainerStarted","Data":"fcfd455d2fc08b264a117df34464fb3fe3d1931ae5aac50eb0129431ff40cbc3"} Jul 13 18:58:57.166364 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.166328 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jczlv" podStartSLOduration=5.587246266 podStartE2EDuration="23.166315658s" totalImagesPullingTime="17.579069392s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:57.1657771 +0000 UTC m=+22.642231292" watchObservedRunningTime="2026-07-13 18:58:57.166315658 +0000 UTC m=+22.642769831" Jul 13 18:58:57.199593 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.199544 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xdbcs" podStartSLOduration=4.583797691 podStartE2EDuration="23.199528609s" totalImagesPullingTime="18.615730918s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:57.199152616 +0000 UTC m=+22.675606809" watchObservedRunningTime="2026-07-13 18:58:57.199528609 +0000 UTC m=+22.675982782" Jul 13 18:58:57.212676 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.212629 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-x6wbt" podStartSLOduration=4.568880905 podStartE2EDuration="23.212611402s" totalImagesPullingTime="18.643730497s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:57.21239991 +0000 UTC m=+22.688854102" watchObservedRunningTime="2026-07-13 18:58:57.212611402 +0000 UTC m=+22.689065576" Jul 13 18:58:57.229943 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.229900 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4pbnd" podStartSLOduration=4.404006679 podStartE2EDuration="23.229885464s" totalImagesPullingTime="18.825878785s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:57.229718217 +0000 UTC m=+22.706172391" watchObservedRunningTime="2026-07-13 18:58:57.229885464 +0000 UTC m=+22.706339638" Jul 13 18:58:57.330762 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.330733 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:58:57.431670 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.431636 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:58:57.535083 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.534918 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:58:57.539290 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.539231 2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 18:58:57.734291 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.734260 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:58:57.834185 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.834097 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:58:57.946053 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.945932 2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T18:58:57.539248284Z","UUID":"1597683e-8977-4f8e-a8ec-28e57f2e87a7","Handler":null,"Name":"","Endpoint":""} Jul 13 18:58:57.947776 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.947753 2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 18:58:57.947922 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.947813 2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 18:58:57.954062 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:57.954031 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:58:58.048954 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:58.048917 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:58:58.049300 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:58.049048 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:58:58.159599 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:58.159447 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" event={"ID":"763b955f-d72c-46af-8046-87d4b90b7f12","Type":"ContainerStarted","Data":"46cda34b2a310badf36e3d67c6d540bfe5c16cfeac20a2ef20cfe740bce33fb3"} Jul 13 18:58:58.162729 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:58.162706 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 18:58:58.163666 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:58.163130 2574 generic.go:356] "Generic (PLEG): container finished" podID="6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7" containerID="3aa36b5a7a6e3b3bfbd4a171f3c79058d401f113ec93f4188f29b470bde8bb8a" exitCode=1 Jul 13 18:58:58.163666 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:58.163495 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" event={"ID":"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7","Type":"ContainerStarted","Data":"b01362eaaefc9c99a6e12a381cb815714582d394fdfddd6343f57166427c01d8"} Jul 13 18:58:58.163666 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:58.163575 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" event={"ID":"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7","Type":"ContainerStarted","Data":"904860b5514b753cc525e6903c811f3e7c0f12599594f4fd8eb5cd53d17ad03c"} Jul 13 18:58:58.163666 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:58.163627 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" event={"ID":"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7","Type":"ContainerStarted","Data":"900acc7356be8eac72f1413fef69071fe268e1644522ecf062edbd468c25dfeb"} Jul 13 18:58:58.163931 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:58.163677 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" event={"ID":"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7","Type":"ContainerStarted","Data":"a544b426b6c403cc57ccc52648a85dcce9414f173400b215df37a776a936ab3c"} Jul 13 18:58:58.163931 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:58.163724 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" event={"ID":"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7","Type":"ContainerDied","Data":"3aa36b5a7a6e3b3bfbd4a171f3c79058d401f113ec93f4188f29b470bde8bb8a"} Jul 13 18:58:58.163931 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:58.163778 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" event={"ID":"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7","Type":"ContainerStarted","Data":"87d871977f1949879eb1e2736049b2bd91f771111c2e4337d2e5ae28bd87bff9"} Jul 13 18:58:59.049377 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:59.049294 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:58:59.049760 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:59.049432 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:58:59.049760 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:59.049505 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:58:59.049760 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:58:59.049633 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:58:59.168013 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:59.167975 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" event={"ID":"763b955f-d72c-46af-8046-87d4b90b7f12","Type":"ContainerStarted","Data":"6e9618460985dd40d1d2f35c0b2fd6775e3488bb7bc13f138d05e0ab96df35fa"} Jul 13 18:58:59.188524 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:59.188474 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmsqw" podStartSLOduration=4.050463826 podStartE2EDuration="25.188459524s" totalImagesPullingTime="21.137995698s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:59.188245668 +0000 UTC m=+24.664699842" watchObservedRunningTime="2026-07-13 18:58:59.188459524 +0000 UTC m=+24.664913695" Jul 13 18:58:59.938677 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:59.938647 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8v8bj" Jul 13 18:58:59.939340 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:58:59.939320 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8v8bj" Jul 13 18:59:00.048682 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:00.048650 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:00.048886 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:00.048804 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:59:01.049160 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:01.049121 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:01.049738 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:01.049170 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:01.049738 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:01.049248 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:59:01.049738 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:01.049358 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:59:02.048618 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:02.048581 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:02.048837 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:02.048751 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:59:02.070718 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:02.070686 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:02.169238 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:02.169204 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:02.176344 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:02.176317 2574 generic.go:356] "Generic (PLEG): container finished" podID="b5a54297-cff5-4224-b5c8-6e1f059d5580" containerID="e79cc08609e6f3a710b82a46874e4469be4fc5c82056277a7caa50d488f27167" exitCode=0 Jul 13 18:59:02.176502 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:02.176382 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qgq48" event={"ID":"b5a54297-cff5-4224-b5c8-6e1f059d5580","Type":"ContainerDied","Data":"e79cc08609e6f3a710b82a46874e4469be4fc5c82056277a7caa50d488f27167"} Jul 13 18:59:02.179594 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:02.179576 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 18:59:02.179975 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:02.179956 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" event={"ID":"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7","Type":"ContainerStarted","Data":"ad12032576afa754e98cc36a2848e59d541865a252a3a6f2d64555f8eba92115"} Jul 13 18:59:02.256565 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:02.256535 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:02.519712 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:02.519629 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:02.846986 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:02.846916 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:02.979894 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:02.979864 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8v8bj" Jul 13 18:59:02.980068 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:02.979991 2574 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 13 18:59:02.980464 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:02.980439 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8v8bj" Jul 13 18:59:03.048350 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:03.048321 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:03.048567 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:03.048325 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:03.048567 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:03.048428 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:59:03.048567 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:03.048510 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:59:03.485556 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:03.485343 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:03.588596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:03.588408 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:04.048831 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:04.048801 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:04.048999 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:04.048902 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:59:04.186711 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:04.186673 2574 generic.go:356] "Generic (PLEG): container finished" podID="b5a54297-cff5-4224-b5c8-6e1f059d5580" containerID="01c3df82fc96d6edd92bb67d29f4af79bfb0dfed54e6908823a0484997b3cfe7" exitCode=0 Jul 13 18:59:04.186883 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:04.186765 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qgq48" event={"ID":"b5a54297-cff5-4224-b5c8-6e1f059d5580","Type":"ContainerDied","Data":"01c3df82fc96d6edd92bb67d29f4af79bfb0dfed54e6908823a0484997b3cfe7"} Jul 13 18:59:04.190211 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:04.190191 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 18:59:04.190567 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:04.190543 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" event={"ID":"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7","Type":"ContainerStarted","Data":"8fad7d437da348c71783a31e0e6ab4d04996363f82584a416394ee91ae27872d"} Jul 13 18:59:04.190820 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:04.190804 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:59:04.190894 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:04.190883 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:59:04.190947 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:04.190937 2574 scope.go:117] "RemoveContainer" containerID="3aa36b5a7a6e3b3bfbd4a171f3c79058d401f113ec93f4188f29b470bde8bb8a" Jul 13 18:59:04.207224 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:04.207203 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:59:04.865189 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:04.865141 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:04.919468 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:04.919417 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:05.049935 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:05.049900 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:05.050100 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:05.049969 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:05.050100 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:05.049982 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:59:05.050100 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:05.050058 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:59:05.200352 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:05.200292 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 18:59:05.200695 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:05.200672 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" event={"ID":"6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7","Type":"ContainerStarted","Data":"8693086ba7fbd9a3383f07f4e87d89485c0c44869d1d89a97af1b7c9eac20a7d"} Jul 13 18:59:05.200947 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:05.200924 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:59:05.216413 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:05.216388 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:59:05.230225 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:05.230175 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" podStartSLOduration=11.379019186 podStartE2EDuration="31.230161359s" totalImagesPullingTime="19.851142173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:05.228762458 +0000 UTC m=+30.705216646" watchObservedRunningTime="2026-07-13 18:59:05.230161359 +0000 UTC m=+30.706615578" Jul 13 18:59:05.334338 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:05.334314 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jczlv_32074cd4-753e-4257-9572-b2c21a540c81/dns-node-resolver/0.log" Jul 13 18:59:05.886635 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:05.886365 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-26wx8"] Jul 13 18:59:05.887129 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:05.886747 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:05.887129 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:05.886895 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:59:05.887291 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:05.887256 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f4ffp"] Jul 13 18:59:05.887385 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:05.887372 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:05.887488 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:05.887466 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:59:05.888040 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:05.888015 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5ftlh"] Jul 13 18:59:05.888134 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:05.888121 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:05.888238 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:05.888218 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:59:05.939902 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:05.939875 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:06.205625 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:06.205588 2574 generic.go:356] "Generic (PLEG): container finished" podID="b5a54297-cff5-4224-b5c8-6e1f059d5580" containerID="880c6e6d31a1f8584617e78cb42a0873b13e468ae67c1084961e09ddb7200c29" exitCode=0 Jul 13 18:59:06.205847 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:06.205649 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qgq48" event={"ID":"b5a54297-cff5-4224-b5c8-6e1f059d5580","Type":"ContainerDied","Data":"880c6e6d31a1f8584617e78cb42a0873b13e468ae67c1084961e09ddb7200c29"} Jul 13 18:59:06.528433 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:06.528342 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:06.535527 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:06.535508 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xdbcs_1989f7e8-7848-4a80-b6f2-2344cdbe418e/node-ca/0.log" Jul 13 18:59:07.048836 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:07.048800 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:07.049499 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:07.048950 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:59:07.098994 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:07.098961 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:07.205602 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:07.205562 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:07.309714 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:07.309621 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:07.418187 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:07.418157 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:07.543362 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:07.543334 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:07.656221 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:07.656185 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:08.048314 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:08.048275 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:08.048480 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:08.048281 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:08.048480 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:08.048422 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:59:08.048602 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:08.048496 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:59:08.155479 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:08.155440 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:08.653731 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:08.653693 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs\") pod \"network-metrics-daemon-26wx8\" (UID: \"b735dd6a-c571-43a2-a6f7-56a9094519a9\") " pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:08.653924 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:08.653740 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret\") pod \"global-pull-secret-syncer-5ftlh\" (UID: \"e8bde243-4f0c-4fd7-a18c-b5900c1609e8\") " pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:08.653924 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:08.653881 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:59:08.653924 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:08.653915 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:59:08.654092 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:08.653959 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs podName:b735dd6a-c571-43a2-a6f7-56a9094519a9 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:40.653938806 +0000 UTC m=+66.130392962 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs") pod "network-metrics-daemon-26wx8" (UID: "b735dd6a-c571-43a2-a6f7-56a9094519a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:59:08.654092 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:08.653986 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret podName:e8bde243-4f0c-4fd7-a18c-b5900c1609e8 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:40.653966487 +0000 UTC m=+66.130420660 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret") pod "global-pull-secret-syncer-5ftlh" (UID: "e8bde243-4f0c-4fd7-a18c-b5900c1609e8") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:59:08.764823 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:08.764773 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:08.855122 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:08.855085 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvrbq\" (UniqueName: \"kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq\") pod \"network-check-target-f4ffp\" (UID: \"b78c80c9-b3b8-40a6-a842-da04dbeddf03\") " pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:08.855301 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:08.855221 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:59:08.855301 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:08.855235 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:59:08.855301 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:08.855244 2574 projected.go:196] Error preparing data for projected volume kube-api-access-vvrbq for pod openshift-network-diagnostics/network-check-target-f4ffp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:59:08.855445 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:08.855305 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq podName:b78c80c9-b3b8-40a6-a842-da04dbeddf03 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:40.855281765 +0000 UTC m=+66.331735918 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vvrbq" (UniqueName: "kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq") pod "network-check-target-f4ffp" (UID: "b78c80c9-b3b8-40a6-a842-da04dbeddf03") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:59:08.974391 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:08.974310 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:09.048994 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:09.048963 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:09.049183 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:09.049118 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:59:10.048353 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:10.048321 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:10.048775 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:10.048455 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:59:10.048775 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:10.048506 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:10.048775 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:10.048622 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:59:10.148649 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:10.148608 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:10.209712 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:10.209676 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:10.368273 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:10.368188 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:10.775486 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:10.775403 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:10.857511 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:10.857478 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:11.048996 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:11.048969 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:11.049475 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:11.049099 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:59:11.218088 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:11.218044 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:11.219412 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:11.219389 2574 generic.go:356] "Generic (PLEG): container finished" podID="b5a54297-cff5-4224-b5c8-6e1f059d5580" containerID="a2c0c689d7118f9d536471e06eb0fb1e0ea7bd9321a224efc98143190333ba20" exitCode=0 Jul 13 18:59:11.219509 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:11.219431 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qgq48" event={"ID":"b5a54297-cff5-4224-b5c8-6e1f059d5580","Type":"ContainerDied","Data":"a2c0c689d7118f9d536471e06eb0fb1e0ea7bd9321a224efc98143190333ba20"} Jul 13 18:59:11.278022 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:11.277992 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:11.559989 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:11.559944 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:12.048511 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:12.048481 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:12.048669 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:12.048526 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:12.048669 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:12.048621 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:59:12.048826 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:12.048731 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:59:12.145363 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:12.145333 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:12.224744 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:12.224710 2574 generic.go:356] "Generic (PLEG): container finished" podID="b5a54297-cff5-4224-b5c8-6e1f059d5580" containerID="2d9a9b42ca67e96bab552b8e54c9b9bf74777bcb2215ffb1579b4bf1155e2ca4" exitCode=0 Jul 13 18:59:12.224890 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:12.224754 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qgq48" event={"ID":"b5a54297-cff5-4224-b5c8-6e1f059d5580","Type":"ContainerDied","Data":"2d9a9b42ca67e96bab552b8e54c9b9bf74777bcb2215ffb1579b4bf1155e2ca4"} Jul 13 18:59:12.296208 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:12.296017 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:12.378904 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:12.378871 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:13.017526 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:13.017496 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:13.048336 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:13.048308 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:13.048457 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:13.048410 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:59:13.230848 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:13.230814 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qgq48" event={"ID":"b5a54297-cff5-4224-b5c8-6e1f059d5580","Type":"ContainerStarted","Data":"62dfc5746528e9b595e00395a59b4553e6cfcd1100715cf313bd798112426b70"} Jul 13 18:59:13.257080 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:13.257031 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qgq48" podStartSLOduration=9.295975409 podStartE2EDuration="39.257017985s" totalImagesPullingTime="29.961042576s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:13.256475931 +0000 UTC m=+38.732930105" watchObservedRunningTime="2026-07-13 18:59:13.257017985 +0000 UTC m=+38.733472188" Jul 13 18:59:13.368096 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:13.368068 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:13.581051 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:13.581017 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:13.978511 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:13.978465 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:14.048672 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:14.048634 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:14.048837 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:14.048684 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:14.048837 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:14.048761 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:59:14.048938 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:14.048867 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:59:15.033656 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:15.033631 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:15.049126 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:15.049094 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:15.049255 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:15.049172 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:59:15.761017 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:15.760989 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:15.861100 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:15.861067 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:15.962690 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:15.962656 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:16.048630 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:16.048605 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:16.048984 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:16.048637 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:16.048984 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:16.048725 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:59:16.048984 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:16.048857 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:59:16.562416 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:16.562381 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:17.048518 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:17.048488 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:17.048672 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:17.048591 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:59:17.187551 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:17.187510 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:17.387727 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:17.387638 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:17.582505 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:17.582471 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:17.782422 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:17.782327 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:17.864122 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:17.864092 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:18.048210 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:18.048185 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:18.048357 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:18.048186 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:18.048400 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:18.048272 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:59:18.048400 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:18.048372 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:59:19.049027 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:19.048985 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:19.049402 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:19.049125 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:59:20.048195 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:20.048158 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:20.048400 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:20.048293 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:59:20.048400 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:20.048341 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:20.048520 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:20.048431 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:59:20.971080 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:20.971051 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:21.044507 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:21.044481 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:21.051290 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:21.051269 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:21.051418 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:21.051368 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:59:22.048355 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:22.048323 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:22.048708 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:22.048323 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:22.048708 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:22.048419 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:59:22.048708 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:22.048540 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:59:23.051644 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:23.051614 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:23.052014 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:23.051708 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:59:24.048302 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:24.048271 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:24.048477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:24.048272 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:24.048477 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:24.048368 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:59:24.048477 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:24.048451 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:59:25.049107 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:25.049077 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:25.049474 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:25.049164 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:59:26.049210 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:26.049173 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:26.049210 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:26.049213 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:26.049627 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:26.049285 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:59:26.049627 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:26.049397 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:59:26.721155 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:26.721106 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:27.048655 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:27.048632 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:27.048810 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:27.048746 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ftlh" podUID="e8bde243-4f0c-4fd7-a18c-b5900c1609e8" Jul 13 18:59:28.049224 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.049191 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:28.049732 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.049192 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:28.049732 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:28.049295 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4ffp" podUID="b78c80c9-b3b8-40a6-a842-da04dbeddf03" Jul 13 18:59:28.049732 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:28.049412 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26wx8" podUID="b735dd6a-c571-43a2-a6f7-56a9094519a9" Jul 13 18:59:28.105762 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.105736 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:28.218379 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.218353 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:28.364596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.364508 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:28.457439 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.457407 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:28.800722 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.800692 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeReady" Jul 13 18:59:28.800920 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.800847 2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 18:59:28.868004 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.867980 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pvjsb"] Jul 13 18:59:28.869860 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.869839 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pvjsb" Jul 13 18:59:28.871024 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.871006 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mdkrq"] Jul 13 18:59:28.872739 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.872722 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mdkrq" Jul 13 18:59:28.874861 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.874810 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 18:59:28.874861 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.874828 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 18:59:28.874861 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.874840 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wt7nz\"" Jul 13 18:59:28.875695 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.875676 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 18:59:28.875794 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.875673 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 18:59:28.875917 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.875775 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mbvx8\"" Jul 13 18:59:28.875917 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.875868 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 18:59:28.886169 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.884547 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pvjsb"] Jul 13 18:59:28.890076 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.890050 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mdkrq"] Jul 13 18:59:28.899937 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.899912 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wvtfn"] Jul 13 18:59:28.901653 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.901637 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wvtfn" Jul 13 18:59:28.904127 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.904107 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 18:59:28.904439 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.904423 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 18:59:28.904602 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.904579 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d2l29\"" Jul 13 18:59:28.904921 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.904906 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 18:59:28.905157 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.905143 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 18:59:28.907637 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.907582 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46876142-cde9-4280-924b-b72711f9af98-config-volume\") pod \"dns-default-pvjsb\" (UID: \"46876142-cde9-4280-924b-b72711f9af98\") " pod="openshift-dns/dns-default-pvjsb" Jul 13 18:59:28.907637 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.907616 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46876142-cde9-4280-924b-b72711f9af98-tmp-dir\") pod \"dns-default-pvjsb\" (UID: \"46876142-cde9-4280-924b-b72711f9af98\") " pod="openshift-dns/dns-default-pvjsb" Jul 13 18:59:28.907759 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.907643 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8e76368-3984-43e3-a888-259acc7cd8d2-cert\") pod \"ingress-canary-mdkrq\" (UID: \"c8e76368-3984-43e3-a888-259acc7cd8d2\") " pod="openshift-ingress-canary/ingress-canary-mdkrq" Jul 13 18:59:28.907759 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.907719 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zjr6\" (UniqueName: \"kubernetes.io/projected/46876142-cde9-4280-924b-b72711f9af98-kube-api-access-8zjr6\") pod \"dns-default-pvjsb\" (UID: \"46876142-cde9-4280-924b-b72711f9af98\") " pod="openshift-dns/dns-default-pvjsb" Jul 13 18:59:28.907759 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.907751 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46876142-cde9-4280-924b-b72711f9af98-metrics-tls\") pod \"dns-default-pvjsb\" (UID: \"46876142-cde9-4280-924b-b72711f9af98\") " pod="openshift-dns/dns-default-pvjsb" Jul 13 18:59:28.907938 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.907803 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52vqs\" (UniqueName: \"kubernetes.io/projected/c8e76368-3984-43e3-a888-259acc7cd8d2-kube-api-access-52vqs\") pod \"ingress-canary-mdkrq\" (UID: \"c8e76368-3984-43e3-a888-259acc7cd8d2\") " pod="openshift-ingress-canary/ingress-canary-mdkrq" Jul 13 18:59:28.917814 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:28.917794 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wvtfn"] Jul 13 18:59:29.009169 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.009133 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/86781ea3-12c5-4237-9a60-a0e514637a8a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvtfn\" (UID: \"86781ea3-12c5-4237-9a60-a0e514637a8a\") " pod="openshift-insights/insights-runtime-extractor-wvtfn" Jul 13 18:59:29.009335 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.009186 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zjr6\" (UniqueName: \"kubernetes.io/projected/46876142-cde9-4280-924b-b72711f9af98-kube-api-access-8zjr6\") pod \"dns-default-pvjsb\" (UID: \"46876142-cde9-4280-924b-b72711f9af98\") " pod="openshift-dns/dns-default-pvjsb" Jul 13 18:59:29.009335 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.009208 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46876142-cde9-4280-924b-b72711f9af98-metrics-tls\") pod \"dns-default-pvjsb\" (UID: \"46876142-cde9-4280-924b-b72711f9af98\") " pod="openshift-dns/dns-default-pvjsb" Jul 13 18:59:29.009335 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.009227 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/86781ea3-12c5-4237-9a60-a0e514637a8a-crio-socket\") pod \"insights-runtime-extractor-wvtfn\" (UID: \"86781ea3-12c5-4237-9a60-a0e514637a8a\") " pod="openshift-insights/insights-runtime-extractor-wvtfn" Jul 13 18:59:29.009335 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.009244 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6mtp\" (UniqueName: \"kubernetes.io/projected/86781ea3-12c5-4237-9a60-a0e514637a8a-kube-api-access-d6mtp\") pod \"insights-runtime-extractor-wvtfn\" (UID: \"86781ea3-12c5-4237-9a60-a0e514637a8a\") " pod="openshift-insights/insights-runtime-extractor-wvtfn" Jul 13 18:59:29.009335 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.009266 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52vqs\" (UniqueName: \"kubernetes.io/projected/c8e76368-3984-43e3-a888-259acc7cd8d2-kube-api-access-52vqs\") pod \"ingress-canary-mdkrq\" (UID: \"c8e76368-3984-43e3-a888-259acc7cd8d2\") " pod="openshift-ingress-canary/ingress-canary-mdkrq" Jul 13 18:59:29.009335 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.009286 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/86781ea3-12c5-4237-9a60-a0e514637a8a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvtfn\" (UID: \"86781ea3-12c5-4237-9a60-a0e514637a8a\") " pod="openshift-insights/insights-runtime-extractor-wvtfn" Jul 13 18:59:29.009335 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.009327 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/86781ea3-12c5-4237-9a60-a0e514637a8a-data-volume\") pod \"insights-runtime-extractor-wvtfn\" (UID: \"86781ea3-12c5-4237-9a60-a0e514637a8a\") " pod="openshift-insights/insights-runtime-extractor-wvtfn" Jul 13 18:59:29.009597 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.009352 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46876142-cde9-4280-924b-b72711f9af98-config-volume\") pod \"dns-default-pvjsb\" (UID: \"46876142-cde9-4280-924b-b72711f9af98\") " pod="openshift-dns/dns-default-pvjsb" Jul 13 18:59:29.009597 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.009376 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46876142-cde9-4280-924b-b72711f9af98-tmp-dir\") pod \"dns-default-pvjsb\" (UID: \"46876142-cde9-4280-924b-b72711f9af98\") " pod="openshift-dns/dns-default-pvjsb" Jul 13 18:59:29.009597 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.009400 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8e76368-3984-43e3-a888-259acc7cd8d2-cert\") pod \"ingress-canary-mdkrq\" (UID: \"c8e76368-3984-43e3-a888-259acc7cd8d2\") " pod="openshift-ingress-canary/ingress-canary-mdkrq" Jul 13 18:59:29.009832 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.009810 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46876142-cde9-4280-924b-b72711f9af98-tmp-dir\") pod \"dns-default-pvjsb\" (UID: \"46876142-cde9-4280-924b-b72711f9af98\") " pod="openshift-dns/dns-default-pvjsb" Jul 13 18:59:29.010045 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.010027 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46876142-cde9-4280-924b-b72711f9af98-config-volume\") pod \"dns-default-pvjsb\" (UID: \"46876142-cde9-4280-924b-b72711f9af98\") " pod="openshift-dns/dns-default-pvjsb" Jul 13 18:59:29.013413 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.013390 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46876142-cde9-4280-924b-b72711f9af98-metrics-tls\") pod \"dns-default-pvjsb\" (UID: \"46876142-cde9-4280-924b-b72711f9af98\") " pod="openshift-dns/dns-default-pvjsb" Jul 13 18:59:29.013535 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.013431 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8e76368-3984-43e3-a888-259acc7cd8d2-cert\") pod \"ingress-canary-mdkrq\" (UID: \"c8e76368-3984-43e3-a888-259acc7cd8d2\") " pod="openshift-ingress-canary/ingress-canary-mdkrq" Jul 13 18:59:29.026443 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.026413 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52vqs\" (UniqueName: \"kubernetes.io/projected/c8e76368-3984-43e3-a888-259acc7cd8d2-kube-api-access-52vqs\") pod \"ingress-canary-mdkrq\" (UID: \"c8e76368-3984-43e3-a888-259acc7cd8d2\") " pod="openshift-ingress-canary/ingress-canary-mdkrq" Jul 13 18:59:29.026554 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.026502 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zjr6\" (UniqueName: \"kubernetes.io/projected/46876142-cde9-4280-924b-b72711f9af98-kube-api-access-8zjr6\") pod \"dns-default-pvjsb\" (UID: \"46876142-cde9-4280-924b-b72711f9af98\") " pod="openshift-dns/dns-default-pvjsb" Jul 13 18:59:29.048499 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.048466 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:29.051244 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.051222 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 18:59:29.091238 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.091213 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:29.110354 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.110318 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/86781ea3-12c5-4237-9a60-a0e514637a8a-data-volume\") pod \"insights-runtime-extractor-wvtfn\" (UID: \"86781ea3-12c5-4237-9a60-a0e514637a8a\") " pod="openshift-insights/insights-runtime-extractor-wvtfn" Jul 13 18:59:29.110494 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.110393 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/86781ea3-12c5-4237-9a60-a0e514637a8a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvtfn\" (UID: \"86781ea3-12c5-4237-9a60-a0e514637a8a\") " pod="openshift-insights/insights-runtime-extractor-wvtfn" Jul 13 18:59:29.110494 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.110438 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/86781ea3-12c5-4237-9a60-a0e514637a8a-crio-socket\") pod \"insights-runtime-extractor-wvtfn\" (UID: \"86781ea3-12c5-4237-9a60-a0e514637a8a\") " pod="openshift-insights/insights-runtime-extractor-wvtfn" Jul 13 18:59:29.110494 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.110465 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6mtp\" (UniqueName: \"kubernetes.io/projected/86781ea3-12c5-4237-9a60-a0e514637a8a-kube-api-access-d6mtp\") pod \"insights-runtime-extractor-wvtfn\" (UID: \"86781ea3-12c5-4237-9a60-a0e514637a8a\") " pod="openshift-insights/insights-runtime-extractor-wvtfn" Jul 13 18:59:29.110630 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.110510 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/86781ea3-12c5-4237-9a60-a0e514637a8a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvtfn\" (UID: \"86781ea3-12c5-4237-9a60-a0e514637a8a\") " pod="openshift-insights/insights-runtime-extractor-wvtfn" Jul 13 18:59:29.110630 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.110526 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/86781ea3-12c5-4237-9a60-a0e514637a8a-crio-socket\") pod \"insights-runtime-extractor-wvtfn\" (UID: \"86781ea3-12c5-4237-9a60-a0e514637a8a\") " pod="openshift-insights/insights-runtime-extractor-wvtfn" Jul 13 18:59:29.110695 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.110641 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/86781ea3-12c5-4237-9a60-a0e514637a8a-data-volume\") pod \"insights-runtime-extractor-wvtfn\" (UID: \"86781ea3-12c5-4237-9a60-a0e514637a8a\") " pod="openshift-insights/insights-runtime-extractor-wvtfn" Jul 13 18:59:29.111004 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.110988 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/86781ea3-12c5-4237-9a60-a0e514637a8a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvtfn\" (UID: \"86781ea3-12c5-4237-9a60-a0e514637a8a\") " pod="openshift-insights/insights-runtime-extractor-wvtfn" Jul 13 18:59:29.112761 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.112746 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/86781ea3-12c5-4237-9a60-a0e514637a8a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvtfn\" (UID: \"86781ea3-12c5-4237-9a60-a0e514637a8a\") " pod="openshift-insights/insights-runtime-extractor-wvtfn" Jul 13 18:59:29.119111 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.119090 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6mtp\" (UniqueName: \"kubernetes.io/projected/86781ea3-12c5-4237-9a60-a0e514637a8a-kube-api-access-d6mtp\") pod \"insights-runtime-extractor-wvtfn\" (UID: \"86781ea3-12c5-4237-9a60-a0e514637a8a\") " pod="openshift-insights/insights-runtime-extractor-wvtfn" Jul 13 18:59:29.159172 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.159085 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:29.180030 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.180009 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pvjsb" Jul 13 18:59:29.188602 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.188457 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mdkrq" Jul 13 18:59:29.210237 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.210210 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wvtfn" Jul 13 18:59:29.339737 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.339707 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pvjsb"] Jul 13 18:59:29.342959 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:59:29.342934 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46876142_cde9_4280_924b_b72711f9af98.slice/crio-6387646e7213c22f369f5887d9e37f9a976555d1f314af91a5909d3bf6e26dd1 WatchSource:0}: Error finding container 6387646e7213c22f369f5887d9e37f9a976555d1f314af91a5909d3bf6e26dd1: Status 404 returned error can't find the container with id 6387646e7213c22f369f5887d9e37f9a976555d1f314af91a5909d3bf6e26dd1 Jul 13 18:59:29.556903 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.556866 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mdkrq"] Jul 13 18:59:29.559597 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:59:29.559568 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e76368_3984_43e3_a888_259acc7cd8d2.slice/crio-a2b56d165ba3f815a2fb073c0302a857155faec69381cda1f4b2437ea0c7e7fc WatchSource:0}: Error finding container a2b56d165ba3f815a2fb073c0302a857155faec69381cda1f4b2437ea0c7e7fc: Status 404 returned error can't find the container with id a2b56d165ba3f815a2fb073c0302a857155faec69381cda1f4b2437ea0c7e7fc Jul 13 18:59:29.561905 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:29.561884 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wvtfn"] Jul 13 18:59:29.564033 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:59:29.564006 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86781ea3_12c5_4237_9a60_a0e514637a8a.slice/crio-34d0a0bb587cb50721f7b1657c6a6edde0c760c2f41478dc3b3218bfd353e37b WatchSource:0}: Error finding container 34d0a0bb587cb50721f7b1657c6a6edde0c760c2f41478dc3b3218bfd353e37b: Status 404 returned error can't find the container with id 34d0a0bb587cb50721f7b1657c6a6edde0c760c2f41478dc3b3218bfd353e37b Jul 13 18:59:30.048952 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:30.048922 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:30.048952 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:30.048948 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:30.052349 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:30.052331 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bqf9d\"" Jul 13 18:59:30.052721 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:30.052368 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 18:59:30.053659 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:30.053564 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 18:59:30.053659 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:30.053635 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 18:59:30.053882 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:30.053651 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cvxxg\"" Jul 13 18:59:30.277903 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:30.277859 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mdkrq" event={"ID":"c8e76368-3984-43e3-a888-259acc7cd8d2","Type":"ContainerStarted","Data":"a2b56d165ba3f815a2fb073c0302a857155faec69381cda1f4b2437ea0c7e7fc"} Jul 13 18:59:30.279352 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:30.279279 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wvtfn" event={"ID":"86781ea3-12c5-4237-9a60-a0e514637a8a","Type":"ContainerStarted","Data":"34d0a0bb587cb50721f7b1657c6a6edde0c760c2f41478dc3b3218bfd353e37b"} Jul 13 18:59:30.280631 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:30.280604 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pvjsb" event={"ID":"46876142-cde9-4280-924b-b72711f9af98","Type":"ContainerStarted","Data":"6387646e7213c22f369f5887d9e37f9a976555d1f314af91a5909d3bf6e26dd1"} Jul 13 18:59:31.286854 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:31.286013 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pvjsb" event={"ID":"46876142-cde9-4280-924b-b72711f9af98","Type":"ContainerStarted","Data":"05686855c6b07c95213abd5504aabd0d50e7e2082b9bc67dab4f2f526a5a65cb"} Jul 13 18:59:31.286854 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:31.286112 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pvjsb" event={"ID":"46876142-cde9-4280-924b-b72711f9af98","Type":"ContainerStarted","Data":"05cc8001c76224c96bc3b4f50a6d64b47b5b2b794285042fa7fb59cd3dd2fd03"} Jul 13 18:59:31.287384 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:31.286934 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pvjsb" Jul 13 18:59:32.182343 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:32.182306 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:32.292950 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:32.292919 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:32.350002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:32.349947 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pvjsb" podStartSLOduration=2.8067156239999997 podStartE2EDuration="4.349927588s" totalImagesPullingTime="1.543211964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:31.308835141 +0000 UTC m=+56.785289315" watchObservedRunningTime="2026-07-13 18:59:32.349927588 +0000 UTC m=+57.826381762" Jul 13 18:59:32.350186 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:32.350160 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bh8vz"] Jul 13 18:59:32.352751 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:32.352688 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bh8vz" Jul 13 18:59:32.354978 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:32.354950 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 13 18:59:32.354978 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:32.354966 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-4nqml\"" Jul 13 18:59:32.365538 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:32.365508 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bh8vz"] Jul 13 18:59:32.366577 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:32.366548 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:32.438016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:32.437924 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8c4174fc-cb32-4f1b-bff6-8193e0d658a5-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bh8vz\" (UID: \"8c4174fc-cb32-4f1b-bff6-8193e0d658a5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bh8vz" Jul 13 18:59:32.539495 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:32.539390 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8c4174fc-cb32-4f1b-bff6-8193e0d658a5-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bh8vz\" (UID: \"8c4174fc-cb32-4f1b-bff6-8193e0d658a5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bh8vz" Jul 13 18:59:32.539495 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:32.539477 2574 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 13 18:59:32.539709 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:32.539557 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c4174fc-cb32-4f1b-bff6-8193e0d658a5-tls-certificates podName:8c4174fc-cb32-4f1b-bff6-8193e0d658a5 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:33.039537965 +0000 UTC m=+58.515992137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/8c4174fc-cb32-4f1b-bff6-8193e0d658a5-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-bh8vz" (UID: "8c4174fc-cb32-4f1b-bff6-8193e0d658a5") : secret "prometheus-operator-admission-webhook-tls" not found Jul 13 18:59:32.598074 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:32.598036 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:33.043172 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:33.043128 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8c4174fc-cb32-4f1b-bff6-8193e0d658a5-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bh8vz\" (UID: \"8c4174fc-cb32-4f1b-bff6-8193e0d658a5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bh8vz" Jul 13 18:59:33.045696 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:33.045663 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8c4174fc-cb32-4f1b-bff6-8193e0d658a5-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bh8vz\" (UID: \"8c4174fc-cb32-4f1b-bff6-8193e0d658a5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bh8vz" Jul 13 18:59:33.107100 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:33.107067 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:33.198696 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:33.198651 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:33.261414 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:33.261367 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bh8vz" Jul 13 18:59:33.826113 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:33.826085 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:33.893217 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:33.893172 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:35.251880 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.251808 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bh8vz"] Jul 13 18:59:35.255690 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:59:35.255659 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c4174fc_cb32_4f1b_bff6_8193e0d658a5.slice/crio-2da7cee5b03cc369afd9be7878aa89eb4159d74c86cd5dab9c444466fe232f66 WatchSource:0}: Error finding container 2da7cee5b03cc369afd9be7878aa89eb4159d74c86cd5dab9c444466fe232f66: Status 404 returned error can't find the container with id 2da7cee5b03cc369afd9be7878aa89eb4159d74c86cd5dab9c444466fe232f66 Jul 13 18:59:35.298902 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.298870 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wvtfn" event={"ID":"86781ea3-12c5-4237-9a60-a0e514637a8a","Type":"ContainerStarted","Data":"9d167cc8e8e48a124b6e1e82204bc0fc94bedd83294131784986d5d71b98a74b"} Jul 13 18:59:35.300188 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.300158 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bh8vz" event={"ID":"8c4174fc-cb32-4f1b-bff6-8193e0d658a5","Type":"ContainerStarted","Data":"2da7cee5b03cc369afd9be7878aa89eb4159d74c86cd5dab9c444466fe232f66"} Jul 13 18:59:35.301793 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.301758 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mdkrq" event={"ID":"c8e76368-3984-43e3-a888-259acc7cd8d2","Type":"ContainerStarted","Data":"e8d68919b7a9fad7f514b743e54174edcf28a21cfaa648210069e4db1e8d7c23"} Jul 13 18:59:35.317598 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.317526 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mdkrq" podStartSLOduration=1.749292786 podStartE2EDuration="7.317505369s" totalImagesPullingTime="5.568212583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:35.316314953 +0000 UTC m=+60.792769163" watchObservedRunningTime="2026-07-13 18:59:35.317505369 +0000 UTC m=+60.793959539" Jul 13 18:59:35.439451 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.439371 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-xpkqs"] Jul 13 18:59:35.441382 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.441365 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-xpkqs" Jul 13 18:59:35.443666 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.443641 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-m2vm6\"" Jul 13 18:59:35.444049 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.444030 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 13 18:59:35.444146 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.444131 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 13 18:59:35.454268 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.454244 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-xpkqs"] Jul 13 18:59:35.470292 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.470266 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:35.540968 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.540939 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:35.562017 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.561994 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrkhz\" (UniqueName: \"kubernetes.io/projected/3decbc78-37dd-41de-840a-eb02d2507656-kube-api-access-mrkhz\") pod \"downloads-68c9cbb497-xpkqs\" (UID: \"3decbc78-37dd-41de-840a-eb02d2507656\") " pod="openshift-console/downloads-68c9cbb497-xpkqs" Jul 13 18:59:35.640264 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.640232 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:35.662458 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.662430 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrkhz\" (UniqueName: \"kubernetes.io/projected/3decbc78-37dd-41de-840a-eb02d2507656-kube-api-access-mrkhz\") pod \"downloads-68c9cbb497-xpkqs\" (UID: \"3decbc78-37dd-41de-840a-eb02d2507656\") " pod="openshift-console/downloads-68c9cbb497-xpkqs" Jul 13 18:59:35.671897 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.671868 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrkhz\" (UniqueName: \"kubernetes.io/projected/3decbc78-37dd-41de-840a-eb02d2507656-kube-api-access-mrkhz\") pod \"downloads-68c9cbb497-xpkqs\" (UID: \"3decbc78-37dd-41de-840a-eb02d2507656\") " pod="openshift-console/downloads-68c9cbb497-xpkqs" Jul 13 18:59:35.741715 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.741646 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:35.750605 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.750579 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-xpkqs" Jul 13 18:59:35.872431 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.872385 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-xpkqs"] Jul 13 18:59:35.874622 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:59:35.874594 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3decbc78_37dd_41de_840a_eb02d2507656.slice/crio-c9eeb118aef6e2eba943b72f2cca5fe5e7cb23fe27027a39746ee64e4b0e9145 WatchSource:0}: Error finding container c9eeb118aef6e2eba943b72f2cca5fe5e7cb23fe27027a39746ee64e4b0e9145: Status 404 returned error can't find the container with id c9eeb118aef6e2eba943b72f2cca5fe5e7cb23fe27027a39746ee64e4b0e9145 Jul 13 18:59:35.921475 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.921446 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:35.988025 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:35.987994 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:36.124588 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:36.124548 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:36.225671 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:36.225571 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:36.306157 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:36.306126 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-xpkqs" event={"ID":"3decbc78-37dd-41de-840a-eb02d2507656","Type":"ContainerStarted","Data":"c9eeb118aef6e2eba943b72f2cca5fe5e7cb23fe27027a39746ee64e4b0e9145"} Jul 13 18:59:36.325174 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:36.325146 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:36.426252 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:36.426216 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:36.727727 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:36.727627 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:36.819838 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:36.819808 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:37.136374 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:37.136166 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:37.218466 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:37.218439 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-79lb6" Jul 13 18:59:37.311400 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:37.311353 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bh8vz" event={"ID":"8c4174fc-cb32-4f1b-bff6-8193e0d658a5","Type":"ContainerStarted","Data":"64bef399f667e8e5a1da71185afda587d920b8ebd724887ea11622f7389fbe22"} Jul 13 18:59:37.311821 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:37.311606 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bh8vz" Jul 13 18:59:37.313830 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:37.313777 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wvtfn" event={"ID":"86781ea3-12c5-4237-9a60-a0e514637a8a","Type":"ContainerStarted","Data":"5f783c3f139582dfaa0726dae3ca0654ca77ec3212d60ce304c434b916d571d3"} Jul 13 18:59:37.316569 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:37.316546 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bh8vz" Jul 13 18:59:37.328539 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:37.328486 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bh8vz" podStartSLOduration=3.478581759 podStartE2EDuration="5.328465208s" totalImagesPullingTime="1.849883449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:37.327525508 +0000 UTC m=+62.803979709" watchObservedRunningTime="2026-07-13 18:59:37.328465208 +0000 UTC m=+62.804919383" Jul 13 18:59:37.354190 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:37.354156 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:37.438538 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:37.438450 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:37.537645 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:37.537604 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:38.355053 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.355014 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:38.441637 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.441604 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-ckj74"] Jul 13 18:59:38.443796 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.443746 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-ckj74" Jul 13 18:59:38.447215 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.447033 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 13 18:59:38.447215 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.447203 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-4vpzs\"" Jul 13 18:59:38.448049 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.448022 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 18:59:38.448179 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.448123 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 18:59:38.448179 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.448025 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 18:59:38.448562 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.448536 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 13 18:59:38.457127 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.457102 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:38.462280 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.462252 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-ckj74"] Jul 13 18:59:38.485433 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.485402 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9fc0bf7d-9486-4251-bc38-e671c0cf563b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-ckj74\" (UID: \"9fc0bf7d-9486-4251-bc38-e671c0cf563b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ckj74" Jul 13 18:59:38.485597 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.485462 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fc0bf7d-9486-4251-bc38-e671c0cf563b-metrics-client-ca\") pod \"prometheus-operator-64785b6675-ckj74\" (UID: \"9fc0bf7d-9486-4251-bc38-e671c0cf563b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ckj74" Jul 13 18:59:38.485649 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.485589 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjwvf\" (UniqueName: \"kubernetes.io/projected/9fc0bf7d-9486-4251-bc38-e671c0cf563b-kube-api-access-jjwvf\") pod \"prometheus-operator-64785b6675-ckj74\" (UID: \"9fc0bf7d-9486-4251-bc38-e671c0cf563b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ckj74" Jul 13 18:59:38.485697 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.485641 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9fc0bf7d-9486-4251-bc38-e671c0cf563b-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-ckj74\" (UID: \"9fc0bf7d-9486-4251-bc38-e671c0cf563b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ckj74" Jul 13 18:59:38.523196 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.523142 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:38.586517 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.586483 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fc0bf7d-9486-4251-bc38-e671c0cf563b-metrics-client-ca\") pod \"prometheus-operator-64785b6675-ckj74\" (UID: \"9fc0bf7d-9486-4251-bc38-e671c0cf563b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ckj74" Jul 13 18:59:38.586689 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.586581 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjwvf\" (UniqueName: \"kubernetes.io/projected/9fc0bf7d-9486-4251-bc38-e671c0cf563b-kube-api-access-jjwvf\") pod \"prometheus-operator-64785b6675-ckj74\" (UID: \"9fc0bf7d-9486-4251-bc38-e671c0cf563b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ckj74" Jul 13 18:59:38.586689 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.586610 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9fc0bf7d-9486-4251-bc38-e671c0cf563b-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-ckj74\" (UID: \"9fc0bf7d-9486-4251-bc38-e671c0cf563b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ckj74" Jul 13 18:59:38.586689 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.586646 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9fc0bf7d-9486-4251-bc38-e671c0cf563b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-ckj74\" (UID: \"9fc0bf7d-9486-4251-bc38-e671c0cf563b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ckj74" Jul 13 18:59:38.587378 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.587318 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fc0bf7d-9486-4251-bc38-e671c0cf563b-metrics-client-ca\") pod \"prometheus-operator-64785b6675-ckj74\" (UID: \"9fc0bf7d-9486-4251-bc38-e671c0cf563b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ckj74" Jul 13 18:59:38.589396 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.589372 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9fc0bf7d-9486-4251-bc38-e671c0cf563b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-ckj74\" (UID: \"9fc0bf7d-9486-4251-bc38-e671c0cf563b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ckj74" Jul 13 18:59:38.589518 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.589397 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9fc0bf7d-9486-4251-bc38-e671c0cf563b-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-ckj74\" (UID: \"9fc0bf7d-9486-4251-bc38-e671c0cf563b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ckj74" Jul 13 18:59:38.600021 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.599967 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjwvf\" (UniqueName: \"kubernetes.io/projected/9fc0bf7d-9486-4251-bc38-e671c0cf563b-kube-api-access-jjwvf\") pod \"prometheus-operator-64785b6675-ckj74\" (UID: \"9fc0bf7d-9486-4251-bc38-e671c0cf563b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ckj74" Jul 13 18:59:38.756437 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:38.756285 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-ckj74" Jul 13 18:59:39.565329 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.565291 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cbbbf849d-gwwnr"] Jul 13 18:59:39.574682 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.574651 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.578825 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.578418 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 18:59:39.578825 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.578516 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 18:59:39.578825 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.578418 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jrnfq\"" Jul 13 18:59:39.578825 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.578655 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 18:59:39.578825 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.578704 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 18:59:39.579198 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.578987 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 18:59:39.580428 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.580124 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cbbbf849d-gwwnr"] Jul 13 18:59:39.693951 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.693899 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99241bde-9e5c-428e-9714-bf8b500410c2-service-ca\") pod \"console-5cbbbf849d-gwwnr\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.693951 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.693949 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99241bde-9e5c-428e-9714-bf8b500410c2-oauth-serving-cert\") pod \"console-5cbbbf849d-gwwnr\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.694198 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.694006 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99241bde-9e5c-428e-9714-bf8b500410c2-console-serving-cert\") pod \"console-5cbbbf849d-gwwnr\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.694198 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.694037 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl58j\" (UniqueName: \"kubernetes.io/projected/99241bde-9e5c-428e-9714-bf8b500410c2-kube-api-access-fl58j\") pod \"console-5cbbbf849d-gwwnr\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.694198 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.694160 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99241bde-9e5c-428e-9714-bf8b500410c2-console-config\") pod \"console-5cbbbf849d-gwwnr\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.694371 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.694231 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99241bde-9e5c-428e-9714-bf8b500410c2-console-oauth-config\") pod \"console-5cbbbf849d-gwwnr\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.794971 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.794936 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl58j\" (UniqueName: \"kubernetes.io/projected/99241bde-9e5c-428e-9714-bf8b500410c2-kube-api-access-fl58j\") pod \"console-5cbbbf849d-gwwnr\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.795135 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.795009 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99241bde-9e5c-428e-9714-bf8b500410c2-console-config\") pod \"console-5cbbbf849d-gwwnr\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.795135 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.795063 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99241bde-9e5c-428e-9714-bf8b500410c2-console-oauth-config\") pod \"console-5cbbbf849d-gwwnr\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.795135 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.795120 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99241bde-9e5c-428e-9714-bf8b500410c2-service-ca\") pod \"console-5cbbbf849d-gwwnr\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.795295 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.795144 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99241bde-9e5c-428e-9714-bf8b500410c2-oauth-serving-cert\") pod \"console-5cbbbf849d-gwwnr\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.795295 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.795172 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99241bde-9e5c-428e-9714-bf8b500410c2-console-serving-cert\") pod \"console-5cbbbf849d-gwwnr\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.795717 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.795698 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99241bde-9e5c-428e-9714-bf8b500410c2-console-config\") pod \"console-5cbbbf849d-gwwnr\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.795905 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.795886 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99241bde-9e5c-428e-9714-bf8b500410c2-oauth-serving-cert\") pod \"console-5cbbbf849d-gwwnr\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.795955 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.795926 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99241bde-9e5c-428e-9714-bf8b500410c2-service-ca\") pod \"console-5cbbbf849d-gwwnr\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.798548 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.798524 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99241bde-9e5c-428e-9714-bf8b500410c2-console-oauth-config\") pod \"console-5cbbbf849d-gwwnr\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.798677 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.798561 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99241bde-9e5c-428e-9714-bf8b500410c2-console-serving-cert\") pod \"console-5cbbbf849d-gwwnr\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.803334 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.803299 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl58j\" (UniqueName: \"kubernetes.io/projected/99241bde-9e5c-428e-9714-bf8b500410c2-kube-api-access-fl58j\") pod \"console-5cbbbf849d-gwwnr\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.887738 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.887266 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 18:59:39.969376 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:39.969349 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-ckj74"] Jul 13 18:59:39.971899 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:59:39.971874 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fc0bf7d_9486_4251_bc38_e671c0cf563b.slice/crio-59033329197ae518bc45de92871ddd52be5a760a4cc9fb890b738191cd9995a7 WatchSource:0}: Error finding container 59033329197ae518bc45de92871ddd52be5a760a4cc9fb890b738191cd9995a7: Status 404 returned error can't find the container with id 59033329197ae518bc45de92871ddd52be5a760a4cc9fb890b738191cd9995a7 Jul 13 18:59:40.038052 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.038024 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cbbbf849d-gwwnr"] Jul 13 18:59:40.040538 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:59:40.040504 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99241bde_9e5c_428e_9714_bf8b500410c2.slice/crio-0f1be58eb3f286a4633b183943d18a5526868f235378332ead510ed1a4a8c978 WatchSource:0}: Error finding container 0f1be58eb3f286a4633b183943d18a5526868f235378332ead510ed1a4a8c978: Status 404 returned error can't find the container with id 0f1be58eb3f286a4633b183943d18a5526868f235378332ead510ed1a4a8c978 Jul 13 18:59:40.327332 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.327287 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-ckj74" event={"ID":"9fc0bf7d-9486-4251-bc38-e671c0cf563b","Type":"ContainerStarted","Data":"59033329197ae518bc45de92871ddd52be5a760a4cc9fb890b738191cd9995a7"} Jul 13 18:59:40.329837 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.329799 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wvtfn" event={"ID":"86781ea3-12c5-4237-9a60-a0e514637a8a","Type":"ContainerStarted","Data":"60980021385bd76ad4941c37be735e18eb0575986c9ae8121cdefb6d5d17e94b"} Jul 13 18:59:40.331248 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.331211 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cbbbf849d-gwwnr" event={"ID":"99241bde-9e5c-428e-9714-bf8b500410c2","Type":"ContainerStarted","Data":"0f1be58eb3f286a4633b183943d18a5526868f235378332ead510ed1a4a8c978"} Jul 13 18:59:40.349203 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.349161 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wvtfn" podStartSLOduration=2.180799576 podStartE2EDuration="12.349146241s" totalImagesPullingTime="10.168346665s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:59:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:40.348479674 +0000 UTC m=+65.824933849" watchObservedRunningTime="2026-07-13 18:59:40.349146241 +0000 UTC m=+65.825600415" Jul 13 18:59:40.703103 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.703013 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs\") pod \"network-metrics-daemon-26wx8\" (UID: \"b735dd6a-c571-43a2-a6f7-56a9094519a9\") " pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:40.703103 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.703067 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret\") pod \"global-pull-secret-syncer-5ftlh\" (UID: \"e8bde243-4f0c-4fd7-a18c-b5900c1609e8\") " pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:40.706083 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.706061 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 18:59:40.706192 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.706137 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 18:59:40.716364 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.716341 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b735dd6a-c571-43a2-a6f7-56a9094519a9-metrics-certs\") pod \"network-metrics-daemon-26wx8\" (UID: \"b735dd6a-c571-43a2-a6f7-56a9094519a9\") " pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:40.716493 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.716345 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8bde243-4f0c-4fd7-a18c-b5900c1609e8-original-pull-secret\") pod \"global-pull-secret-syncer-5ftlh\" (UID: \"e8bde243-4f0c-4fd7-a18c-b5900c1609e8\") " pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:40.757483 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.757448 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ftlh" Jul 13 18:59:40.863735 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.863703 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cvxxg\"" Jul 13 18:59:40.871830 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.871803 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26wx8" Jul 13 18:59:40.905199 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.904708 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvrbq\" (UniqueName: \"kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq\") pod \"network-check-target-f4ffp\" (UID: \"b78c80c9-b3b8-40a6-a842-da04dbeddf03\") " pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:40.908196 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.907954 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 18:59:40.918679 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.918432 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 18:59:40.931396 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.931365 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvrbq\" (UniqueName: \"kubernetes.io/projected/b78c80c9-b3b8-40a6-a842-da04dbeddf03-kube-api-access-vvrbq\") pod \"network-check-target-f4ffp\" (UID: \"b78c80c9-b3b8-40a6-a842-da04dbeddf03\") " pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:40.933556 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:40.933466 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5ftlh"] Jul 13 18:59:40.938155 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:59:40.938069 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8bde243_4f0c_4fd7_a18c_b5900c1609e8.slice/crio-c9456f50a3390c40f812940090d69dddfdc18b5138fafac8d0f4d0f8355b7b85 WatchSource:0}: Error finding container c9456f50a3390c40f812940090d69dddfdc18b5138fafac8d0f4d0f8355b7b85: Status 404 returned error can't find the container with id c9456f50a3390c40f812940090d69dddfdc18b5138fafac8d0f4d0f8355b7b85 Jul 13 18:59:41.058095 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:41.058062 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-26wx8"] Jul 13 18:59:41.062761 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:59:41.062730 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb735dd6a_c571_43a2_a6f7_56a9094519a9.slice/crio-f0918fce3afef88d125c4ffaa4a7187efbebccf639bff1f2508a072f02b7a207 WatchSource:0}: Error finding container f0918fce3afef88d125c4ffaa4a7187efbebccf639bff1f2508a072f02b7a207: Status 404 returned error can't find the container with id f0918fce3afef88d125c4ffaa4a7187efbebccf639bff1f2508a072f02b7a207 Jul 13 18:59:41.169154 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:41.169121 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bqf9d\"" Jul 13 18:59:41.176966 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:41.176927 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:41.339939 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:41.339855 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-26wx8" event={"ID":"b735dd6a-c571-43a2-a6f7-56a9094519a9","Type":"ContainerStarted","Data":"f0918fce3afef88d125c4ffaa4a7187efbebccf639bff1f2508a072f02b7a207"} Jul 13 18:59:41.345438 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:41.345401 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5ftlh" event={"ID":"e8bde243-4f0c-4fd7-a18c-b5900c1609e8","Type":"ContainerStarted","Data":"c9456f50a3390c40f812940090d69dddfdc18b5138fafac8d0f4d0f8355b7b85"} Jul 13 18:59:41.362804 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:41.362752 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f4ffp"] Jul 13 18:59:41.367552 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:59:41.367520 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb78c80c9_b3b8_40a6_a842_da04dbeddf03.slice/crio-6ef9416760ad1c29fcafbf0642d56fb11a7f0d8bf5bfc891f9b4ddd20ba2d00b WatchSource:0}: Error finding container 6ef9416760ad1c29fcafbf0642d56fb11a7f0d8bf5bfc891f9b4ddd20ba2d00b: Status 404 returned error can't find the container with id 6ef9416760ad1c29fcafbf0642d56fb11a7f0d8bf5bfc891f9b4ddd20ba2d00b Jul 13 18:59:42.230674 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:42.230639 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:42.295512 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:42.294837 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pvjsb" Jul 13 18:59:42.307825 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:42.307747 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:42.353612 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:42.353540 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f4ffp" event={"ID":"b78c80c9-b3b8-40a6-a842-da04dbeddf03","Type":"ContainerStarted","Data":"6ef9416760ad1c29fcafbf0642d56fb11a7f0d8bf5bfc891f9b4ddd20ba2d00b"} Jul 13 18:59:43.117824 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:43.117716 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:43.366366 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:43.365728 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-ckj74" event={"ID":"9fc0bf7d-9486-4251-bc38-e671c0cf563b","Type":"ContainerStarted","Data":"0d6906d6162eb10ddb1f78a3bfc16b753ee7941382dcdff680f36dc9b59539ca"} Jul 13 18:59:43.366366 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:43.365878 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-ckj74" event={"ID":"9fc0bf7d-9486-4251-bc38-e671c0cf563b","Type":"ContainerStarted","Data":"7c6d913781e82b6efaaf265464307f2ea6e3417e8396a02a86c81fce5b570437"} Jul 13 18:59:43.369819 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:43.369435 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-26wx8" event={"ID":"b735dd6a-c571-43a2-a6f7-56a9094519a9","Type":"ContainerStarted","Data":"f19db078d245f2ff7c0dcf491c24c65ce5c6972ab77495f1f6e24cd1bf429c22"} Jul 13 18:59:43.369819 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:43.369490 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-26wx8" event={"ID":"b735dd6a-c571-43a2-a6f7-56a9094519a9","Type":"ContainerStarted","Data":"a6a0df533a5e5d3fe2fd9870e5c42688eae44209d59360810873655717c39cc5"} Jul 13 18:59:43.386750 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:43.386515 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-ckj74" podStartSLOduration=2.658053377 podStartE2EDuration="5.386494302s" totalImagesPullingTime="2.728440925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:43.384984164 +0000 UTC m=+68.861438339" watchObservedRunningTime="2026-07-13 18:59:43.386494302 +0000 UTC m=+68.862948478" Jul 13 18:59:43.404834 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:43.404748 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-26wx8" podStartSLOduration=67.764921642 podStartE2EDuration="1m9.404726566s" totalImagesPullingTime="1.639804924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:43.404036457 +0000 UTC m=+68.880490632" watchObservedRunningTime="2026-07-13 18:59:43.404726566 +0000 UTC m=+68.881180740" Jul 13 18:59:43.515342 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:43.515295 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:43.614411 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:43.614338 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:44.368775 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.367586 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f56c8fc58-wdt69"] Jul 13 18:59:44.373631 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.373608 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.379206 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.379178 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:44.382572 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.382535 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 18:59:44.382685 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.382587 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f56c8fc58-wdt69"] Jul 13 18:59:44.432548 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.432341 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-oauth-serving-cert\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.432548 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.432419 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-trusted-ca-bundle\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.432548 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.432491 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-console-config\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.432865 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.432579 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-service-ca\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.432865 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.432673 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c8bb84e-6492-4673-a8e1-d20f38615d40-console-oauth-config\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.432865 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.432718 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkdw2\" (UniqueName: \"kubernetes.io/projected/8c8bb84e-6492-4673-a8e1-d20f38615d40-kube-api-access-wkdw2\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.432865 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.432757 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c8bb84e-6492-4673-a8e1-d20f38615d40-console-serving-cert\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.473894 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.473713 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:44.534083 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.534040 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-oauth-serving-cert\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.534273 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.534122 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-trusted-ca-bundle\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.534273 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.534169 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-console-config\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.534273 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.534202 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-service-ca\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.534273 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.534266 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c8bb84e-6492-4673-a8e1-d20f38615d40-console-oauth-config\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.534410 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.534305 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkdw2\" (UniqueName: \"kubernetes.io/projected/8c8bb84e-6492-4673-a8e1-d20f38615d40-kube-api-access-wkdw2\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.534410 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.534348 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c8bb84e-6492-4673-a8e1-d20f38615d40-console-serving-cert\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.537918 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.537287 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-trusted-ca-bundle\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.538142 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.538041 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-oauth-serving-cert\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.538808 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.538670 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-service-ca\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.539395 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.539327 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-console-config\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.542252 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.540852 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c8bb84e-6492-4673-a8e1-d20f38615d40-console-serving-cert\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.545444 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.545399 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c8bb84e-6492-4673-a8e1-d20f38615d40-console-oauth-config\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.551424 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.551374 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkdw2\" (UniqueName: \"kubernetes.io/projected/8c8bb84e-6492-4673-a8e1-d20f38615d40-kube-api-access-wkdw2\") pod \"console-7f56c8fc58-wdt69\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:44.688486 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:44.688399 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 18:59:45.536448 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.536409 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:45.640314 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.640218 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:45.789038 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.789009 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:45.796981 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.796953 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr"] Jul 13 18:59:45.799914 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.799504 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" Jul 13 18:59:45.802806 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.802526 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8kqlr"] Jul 13 18:59:45.803219 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.803200 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 13 18:59:45.803650 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.803622 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 13 18:59:45.804212 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.804166 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-c9qbz\"" Jul 13 18:59:45.805110 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.805090 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-99zld"] Jul 13 18:59:45.805225 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.805211 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:45.807389 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.807358 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 18:59:45.807486 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.807428 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:45.807768 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.807749 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 18:59:45.808061 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.807862 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 18:59:45.808533 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.808403 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-486zq\"" Jul 13 18:59:45.810104 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.810087 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 13 18:59:45.810501 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.810325 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 13 18:59:45.810501 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.810330 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 13 18:59:45.810501 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.810402 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-x894w\"" Jul 13 18:59:45.814126 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.814064 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr"] Jul 13 18:59:45.824263 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.824240 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-99zld"] Jul 13 18:59:45.946486 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.946439 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ca24210-be70-483b-9893-eece94d2756b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:45.946674 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.946488 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phcps\" (UniqueName: \"kubernetes.io/projected/4ca24210-be70-483b-9893-eece94d2756b-kube-api-access-phcps\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:45.946674 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.946520 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f3a9e5d-3247-474e-85fb-1026a2779ee1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-99zld\" (UID: \"3f3a9e5d-3247-474e-85fb-1026a2779ee1\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:45.946674 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.946549 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh7ps\" (UniqueName: \"kubernetes.io/projected/3f3a9e5d-3247-474e-85fb-1026a2779ee1-kube-api-access-rh7ps\") pod \"kube-state-metrics-785778ff74-99zld\" (UID: \"3f3a9e5d-3247-474e-85fb-1026a2779ee1\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:45.946674 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.946571 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ca24210-be70-483b-9893-eece94d2756b-node-exporter-textfile\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:45.946674 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.946628 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ca24210-be70-483b-9893-eece94d2756b-metrics-client-ca\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:45.946674 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.946655 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3f3a9e5d-3247-474e-85fb-1026a2779ee1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-99zld\" (UID: \"3f3a9e5d-3247-474e-85fb-1026a2779ee1\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:45.947048 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.946706 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5g9q\" (UniqueName: \"kubernetes.io/projected/62c27f1a-06ed-42aa-a152-e96e9fc07478-kube-api-access-s5g9q\") pod \"openshift-state-metrics-84bd68b896-b2htr\" (UID: \"62c27f1a-06ed-42aa-a152-e96e9fc07478\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" Jul 13 18:59:45.947048 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.946741 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ca24210-be70-483b-9893-eece94d2756b-sys\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:45.947048 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.946775 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ca24210-be70-483b-9893-eece94d2756b-root\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:45.947048 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.946824 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ca24210-be70-483b-9893-eece94d2756b-node-exporter-tls\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:45.947048 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.946863 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ca24210-be70-483b-9893-eece94d2756b-node-exporter-accelerators-collector-config\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:45.947048 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.946891 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3f3a9e5d-3247-474e-85fb-1026a2779ee1-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-99zld\" (UID: \"3f3a9e5d-3247-474e-85fb-1026a2779ee1\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:45.947048 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.946965 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f3a9e5d-3247-474e-85fb-1026a2779ee1-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-99zld\" (UID: \"3f3a9e5d-3247-474e-85fb-1026a2779ee1\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:45.947048 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.947025 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c27f1a-06ed-42aa-a152-e96e9fc07478-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-b2htr\" (UID: \"62c27f1a-06ed-42aa-a152-e96e9fc07478\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" Jul 13 18:59:45.947048 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.947050 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f3a9e5d-3247-474e-85fb-1026a2779ee1-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-99zld\" (UID: \"3f3a9e5d-3247-474e-85fb-1026a2779ee1\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:45.947477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.947078 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62c27f1a-06ed-42aa-a152-e96e9fc07478-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-b2htr\" (UID: \"62c27f1a-06ed-42aa-a152-e96e9fc07478\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" Jul 13 18:59:45.947477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.947109 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62c27f1a-06ed-42aa-a152-e96e9fc07478-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-b2htr\" (UID: \"62c27f1a-06ed-42aa-a152-e96e9fc07478\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" Jul 13 18:59:45.947477 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:45.947166 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ca24210-be70-483b-9893-eece94d2756b-node-exporter-wtmp\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.048648 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.048491 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f3a9e5d-3247-474e-85fb-1026a2779ee1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-99zld\" (UID: \"3f3a9e5d-3247-474e-85fb-1026a2779ee1\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:46.048648 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.048541 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh7ps\" (UniqueName: \"kubernetes.io/projected/3f3a9e5d-3247-474e-85fb-1026a2779ee1-kube-api-access-rh7ps\") pod \"kube-state-metrics-785778ff74-99zld\" (UID: \"3f3a9e5d-3247-474e-85fb-1026a2779ee1\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:46.048648 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.048577 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ca24210-be70-483b-9893-eece94d2756b-node-exporter-textfile\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.048648 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.048615 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ca24210-be70-483b-9893-eece94d2756b-metrics-client-ca\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.049040 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.048738 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3f3a9e5d-3247-474e-85fb-1026a2779ee1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-99zld\" (UID: \"3f3a9e5d-3247-474e-85fb-1026a2779ee1\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:46.049040 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.048794 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5g9q\" (UniqueName: \"kubernetes.io/projected/62c27f1a-06ed-42aa-a152-e96e9fc07478-kube-api-access-s5g9q\") pod \"openshift-state-metrics-84bd68b896-b2htr\" (UID: \"62c27f1a-06ed-42aa-a152-e96e9fc07478\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" Jul 13 18:59:46.049040 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.048832 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ca24210-be70-483b-9893-eece94d2756b-sys\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.049040 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.048857 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ca24210-be70-483b-9893-eece94d2756b-root\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.049040 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.048896 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ca24210-be70-483b-9893-eece94d2756b-node-exporter-tls\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.049040 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.048942 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ca24210-be70-483b-9893-eece94d2756b-node-exporter-accelerators-collector-config\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.049040 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.048970 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ca24210-be70-483b-9893-eece94d2756b-sys\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.049040 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.048970 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ca24210-be70-483b-9893-eece94d2756b-node-exporter-textfile\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.049040 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.048977 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3f3a9e5d-3247-474e-85fb-1026a2779ee1-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-99zld\" (UID: \"3f3a9e5d-3247-474e-85fb-1026a2779ee1\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:46.049482 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:46.049064 2574 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 13 18:59:46.049482 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:46.049158 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ca24210-be70-483b-9893-eece94d2756b-node-exporter-tls podName:4ca24210-be70-483b-9893-eece94d2756b nodeName:}" failed. No retries permitted until 2026-07-13 18:59:46.54913506 +0000 UTC m=+72.025589222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4ca24210-be70-483b-9893-eece94d2756b-node-exporter-tls") pod "node-exporter-8kqlr" (UID: "4ca24210-be70-483b-9893-eece94d2756b") : secret "node-exporter-tls" not found Jul 13 18:59:46.049482 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.049168 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ca24210-be70-483b-9893-eece94d2756b-root\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.049482 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.049183 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f3a9e5d-3247-474e-85fb-1026a2779ee1-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-99zld\" (UID: \"3f3a9e5d-3247-474e-85fb-1026a2779ee1\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:46.049482 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.049228 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c27f1a-06ed-42aa-a152-e96e9fc07478-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-b2htr\" (UID: \"62c27f1a-06ed-42aa-a152-e96e9fc07478\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" Jul 13 18:59:46.049482 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.049252 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f3a9e5d-3247-474e-85fb-1026a2779ee1-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-99zld\" (UID: \"3f3a9e5d-3247-474e-85fb-1026a2779ee1\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:46.049482 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.049269 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ca24210-be70-483b-9893-eece94d2756b-metrics-client-ca\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.049482 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.049283 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62c27f1a-06ed-42aa-a152-e96e9fc07478-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-b2htr\" (UID: \"62c27f1a-06ed-42aa-a152-e96e9fc07478\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" Jul 13 18:59:46.049482 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.049338 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62c27f1a-06ed-42aa-a152-e96e9fc07478-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-b2htr\" (UID: \"62c27f1a-06ed-42aa-a152-e96e9fc07478\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" Jul 13 18:59:46.049482 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:46.049364 2574 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 13 18:59:46.049482 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.049395 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ca24210-be70-483b-9893-eece94d2756b-node-exporter-wtmp\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.049482 ip-10-0-128-30 kubenswrapper[2574]: E0713 18:59:46.049414 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62c27f1a-06ed-42aa-a152-e96e9fc07478-openshift-state-metrics-tls podName:62c27f1a-06ed-42aa-a152-e96e9fc07478 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:46.549399632 +0000 UTC m=+72.025853800 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/62c27f1a-06ed-42aa-a152-e96e9fc07478-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-b2htr" (UID: "62c27f1a-06ed-42aa-a152-e96e9fc07478") : secret "openshift-state-metrics-tls" not found Jul 13 18:59:46.049482 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.049441 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ca24210-be70-483b-9893-eece94d2756b-node-exporter-accelerators-collector-config\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.049482 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.049453 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ca24210-be70-483b-9893-eece94d2756b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.050158 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.049493 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phcps\" (UniqueName: \"kubernetes.io/projected/4ca24210-be70-483b-9893-eece94d2756b-kube-api-access-phcps\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.050158 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.049519 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ca24210-be70-483b-9893-eece94d2756b-node-exporter-wtmp\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.050158 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.049521 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3f3a9e5d-3247-474e-85fb-1026a2779ee1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-99zld\" (UID: \"3f3a9e5d-3247-474e-85fb-1026a2779ee1\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:46.050158 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.050016 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3f3a9e5d-3247-474e-85fb-1026a2779ee1-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-99zld\" (UID: \"3f3a9e5d-3247-474e-85fb-1026a2779ee1\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:46.050397 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.050370 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62c27f1a-06ed-42aa-a152-e96e9fc07478-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-b2htr\" (UID: \"62c27f1a-06ed-42aa-a152-e96e9fc07478\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" Jul 13 18:59:46.050567 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.050511 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f3a9e5d-3247-474e-85fb-1026a2779ee1-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-99zld\" (UID: \"3f3a9e5d-3247-474e-85fb-1026a2779ee1\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:46.052279 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.052252 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f3a9e5d-3247-474e-85fb-1026a2779ee1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-99zld\" (UID: \"3f3a9e5d-3247-474e-85fb-1026a2779ee1\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:46.052395 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.052278 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ca24210-be70-483b-9893-eece94d2756b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.052659 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.052632 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62c27f1a-06ed-42aa-a152-e96e9fc07478-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-b2htr\" (UID: \"62c27f1a-06ed-42aa-a152-e96e9fc07478\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" Jul 13 18:59:46.053329 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.053306 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f3a9e5d-3247-474e-85fb-1026a2779ee1-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-99zld\" (UID: \"3f3a9e5d-3247-474e-85fb-1026a2779ee1\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:46.069176 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.069118 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phcps\" (UniqueName: \"kubernetes.io/projected/4ca24210-be70-483b-9893-eece94d2756b-kube-api-access-phcps\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.080520 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.080491 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5g9q\" (UniqueName: \"kubernetes.io/projected/62c27f1a-06ed-42aa-a152-e96e9fc07478-kube-api-access-s5g9q\") pod \"openshift-state-metrics-84bd68b896-b2htr\" (UID: \"62c27f1a-06ed-42aa-a152-e96e9fc07478\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" Jul 13 18:59:46.087604 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.087568 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh7ps\" (UniqueName: \"kubernetes.io/projected/3f3a9e5d-3247-474e-85fb-1026a2779ee1-kube-api-access-rh7ps\") pod \"kube-state-metrics-785778ff74-99zld\" (UID: \"3f3a9e5d-3247-474e-85fb-1026a2779ee1\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:46.132497 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.132445 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" Jul 13 18:59:46.519106 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.518985 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:46.554518 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.554485 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ca24210-be70-483b-9893-eece94d2756b-node-exporter-tls\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.555047 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.554581 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c27f1a-06ed-42aa-a152-e96e9fc07478-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-b2htr\" (UID: \"62c27f1a-06ed-42aa-a152-e96e9fc07478\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" Jul 13 18:59:46.557056 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.557004 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ca24210-be70-483b-9893-eece94d2756b-node-exporter-tls\") pod \"node-exporter-8kqlr\" (UID: \"4ca24210-be70-483b-9893-eece94d2756b\") " pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.557381 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.557355 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c27f1a-06ed-42aa-a152-e96e9fc07478-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-b2htr\" (UID: \"62c27f1a-06ed-42aa-a152-e96e9fc07478\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" Jul 13 18:59:46.619686 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.619607 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:46.715765 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.715732 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" Jul 13 18:59:46.727210 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.727180 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8kqlr" Jul 13 18:59:46.760419 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.760343 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:46.860829 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.860793 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:46.867095 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.867064 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 18:59:46.871102 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.871073 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:46.873502 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.873481 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 13 18:59:46.873818 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.873681 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 13 18:59:46.873927 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.873912 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 13 18:59:46.874114 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.874096 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 13 18:59:46.874287 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.874274 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 13 18:59:46.874469 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.874454 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4jwgl\"" Jul 13 18:59:46.874702 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.874677 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 13 18:59:46.874842 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.874755 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 13 18:59:46.874937 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.874927 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 13 18:59:46.876912 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.876871 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 13 18:59:46.889152 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.889130 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 18:59:46.958516 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.958475 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-web-config\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:46.958687 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.958528 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:46.958687 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.958560 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:46.958687 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.958678 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a720e726-5e5b-44c4-83c7-06f5c5aa4554-config-out\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:46.958923 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.958712 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-config-volume\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:46.958923 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.958738 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a720e726-5e5b-44c4-83c7-06f5c5aa4554-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:46.958923 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.958808 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:46.958923 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.958856 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:46.959131 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.958945 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4nz4\" (UniqueName: \"kubernetes.io/projected/a720e726-5e5b-44c4-83c7-06f5c5aa4554-kube-api-access-d4nz4\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:46.959131 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.958988 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a720e726-5e5b-44c4-83c7-06f5c5aa4554-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:46.959131 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.959039 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a720e726-5e5b-44c4-83c7-06f5c5aa4554-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:46.959131 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.959072 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:46.959131 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.959115 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a720e726-5e5b-44c4-83c7-06f5c5aa4554-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:46.960386 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:46.960340 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:47.060551 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.059986 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a720e726-5e5b-44c4-83c7-06f5c5aa4554-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.060551 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.060032 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.060551 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.060061 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a720e726-5e5b-44c4-83c7-06f5c5aa4554-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.060551 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.060094 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-web-config\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.060551 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.060125 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.060551 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.060173 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.060551 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.060220 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a720e726-5e5b-44c4-83c7-06f5c5aa4554-config-out\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.060551 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.060244 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-config-volume\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.060551 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.060268 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a720e726-5e5b-44c4-83c7-06f5c5aa4554-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.060551 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.060297 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.060551 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.060343 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.060551 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.060370 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4nz4\" (UniqueName: \"kubernetes.io/projected/a720e726-5e5b-44c4-83c7-06f5c5aa4554-kube-api-access-d4nz4\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.060551 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.060397 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a720e726-5e5b-44c4-83c7-06f5c5aa4554-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.061395 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.061032 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a720e726-5e5b-44c4-83c7-06f5c5aa4554-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.061822 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.061455 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a720e726-5e5b-44c4-83c7-06f5c5aa4554-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.062695 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.062661 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a720e726-5e5b-44c4-83c7-06f5c5aa4554-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.064685 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.064598 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.064964 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.064936 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a720e726-5e5b-44c4-83c7-06f5c5aa4554-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.065097 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.065077 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.065173 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.065152 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-config-volume\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.065524 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.065501 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.066564 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.066538 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a720e726-5e5b-44c4-83c7-06f5c5aa4554-config-out\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.066656 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.066570 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-web-config\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.067117 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.067092 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.068178 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.068135 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.069993 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.069971 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4nz4\" (UniqueName: \"kubernetes.io/projected/a720e726-5e5b-44c4-83c7-06f5c5aa4554-kube-api-access-d4nz4\") pod \"alertmanager-main-0\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.186469 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.186376 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 18:59:47.838881 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.838841 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:47.939016 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:47.938926 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:48.736995 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.736967 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:48.803997 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.803896 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:48.893457 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.893421 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-b7c576458-z48dk"] Jul 13 18:59:48.898436 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.898406 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:48.901192 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.901159 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 13 18:59:48.901319 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.901166 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 13 18:59:48.901319 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.901257 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 13 18:59:48.901429 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.901169 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 13 18:59:48.901660 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.901643 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 13 18:59:48.901749 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.901682 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-5tqq6\"" Jul 13 18:59:48.901828 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.901765 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6vhng2va5rone\"" Jul 13 18:59:48.912923 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.912903 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b7c576458-z48dk"] Jul 13 18:59:48.919360 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.919325 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:48.980459 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.977157 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/980c5a95-d03c-4018-9432-2db17d31c30b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:48.980459 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.977222 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/980c5a95-d03c-4018-9432-2db17d31c30b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:48.980459 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.977299 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/980c5a95-d03c-4018-9432-2db17d31c30b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:48.980459 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.977333 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/980c5a95-d03c-4018-9432-2db17d31c30b-secret-grpc-tls\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:48.980459 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.977381 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/980c5a95-d03c-4018-9432-2db17d31c30b-metrics-client-ca\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:48.980459 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.977446 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqnr2\" (UniqueName: \"kubernetes.io/projected/980c5a95-d03c-4018-9432-2db17d31c30b-kube-api-access-kqnr2\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:48.980459 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.977494 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/980c5a95-d03c-4018-9432-2db17d31c30b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:48.980459 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:48.977539 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/980c5a95-d03c-4018-9432-2db17d31c30b-secret-thanos-querier-tls\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:49.078705 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:49.078665 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqnr2\" (UniqueName: \"kubernetes.io/projected/980c5a95-d03c-4018-9432-2db17d31c30b-kube-api-access-kqnr2\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:49.078924 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:49.078725 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/980c5a95-d03c-4018-9432-2db17d31c30b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:49.078924 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:49.078759 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/980c5a95-d03c-4018-9432-2db17d31c30b-secret-thanos-querier-tls\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:49.078924 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:49.078796 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/980c5a95-d03c-4018-9432-2db17d31c30b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:49.078924 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:49.078817 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/980c5a95-d03c-4018-9432-2db17d31c30b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:49.078924 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:49.078856 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/980c5a95-d03c-4018-9432-2db17d31c30b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:49.078924 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:49.078883 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/980c5a95-d03c-4018-9432-2db17d31c30b-secret-grpc-tls\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:49.078924 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:49.078922 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/980c5a95-d03c-4018-9432-2db17d31c30b-metrics-client-ca\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:49.080172 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:49.080140 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/980c5a95-d03c-4018-9432-2db17d31c30b-metrics-client-ca\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:49.082251 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:49.082015 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/980c5a95-d03c-4018-9432-2db17d31c30b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:49.082251 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:49.082127 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/980c5a95-d03c-4018-9432-2db17d31c30b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:49.082251 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:49.082211 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/980c5a95-d03c-4018-9432-2db17d31c30b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:49.082470 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:49.082447 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/980c5a95-d03c-4018-9432-2db17d31c30b-secret-grpc-tls\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:49.082532 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:49.082498 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/980c5a95-d03c-4018-9432-2db17d31c30b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:49.082596 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:49.082575 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/980c5a95-d03c-4018-9432-2db17d31c30b-secret-thanos-querier-tls\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:49.087333 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:49.087312 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqnr2\" (UniqueName: \"kubernetes.io/projected/980c5a95-d03c-4018-9432-2db17d31c30b-kube-api-access-kqnr2\") pod \"thanos-querier-b7c576458-z48dk\" (UID: \"980c5a95-d03c-4018-9432-2db17d31c30b\") " pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:49.209569 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:49.209532 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 18:59:49.564469 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:49.564430 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:50.222711 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.222676 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-79664f77cb-c64s2"] Jul 13 18:59:50.227778 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.227747 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.230194 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.230163 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 13 18:59:50.231079 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.231053 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-nk7v4\"" Jul 13 18:59:50.231210 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.231116 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 13 18:59:50.231210 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.231142 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 13 18:59:50.231325 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.231305 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 13 18:59:50.231396 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.231381 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6kefl3ir9b60f\"" Jul 13 18:59:50.235201 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.235089 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79664f77cb-c64s2"] Jul 13 18:59:50.290430 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.290389 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-metrics-server-audit-profiles\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.290611 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.290451 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-secret-metrics-server-tls\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.290611 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.290492 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-secret-metrics-server-client-certs\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.290611 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.290526 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-client-ca-bundle\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.290611 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.290548 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spgsz\" (UniqueName: \"kubernetes.io/projected/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-kube-api-access-spgsz\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.290611 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.290605 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-audit-log\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.290867 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.290684 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.392051 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.392014 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-audit-log\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.392256 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.392081 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.392256 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.392117 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-metrics-server-audit-profiles\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.392256 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.392152 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-secret-metrics-server-tls\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.392256 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.392183 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-secret-metrics-server-client-certs\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.392468 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.392276 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-client-ca-bundle\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.392468 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.392309 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spgsz\" (UniqueName: \"kubernetes.io/projected/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-kube-api-access-spgsz\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.392468 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.392416 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-audit-log\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.393193 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.392737 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.393549 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.393524 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-metrics-server-audit-profiles\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.395291 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.395266 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-secret-metrics-server-tls\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.395487 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.395466 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-secret-metrics-server-client-certs\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.395570 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.395530 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-client-ca-bundle\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.400569 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.400544 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spgsz\" (UniqueName: \"kubernetes.io/projected/eaa1640a-5f83-4a1a-aa77-fbcbb321acb6-kube-api-access-spgsz\") pod \"metrics-server-79664f77cb-c64s2\" (UID: \"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6\") " pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.541132 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.541100 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 18:59:50.557815 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.557763 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:50.567201 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.567180 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-4m48s"] Jul 13 18:59:50.570800 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.570770 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-4m48s" Jul 13 18:59:50.573268 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.573236 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 13 18:59:50.573385 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.573307 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-h6scx\"" Jul 13 18:59:50.578089 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.578061 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cbbbf849d-gwwnr"] Jul 13 18:59:50.582457 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.582436 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-4m48s"] Jul 13 18:59:50.612501 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.612461 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7755b5d74f-gj8g9"] Jul 13 18:59:50.618304 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.618267 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.631675 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.631591 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7755b5d74f-gj8g9"] Jul 13 18:59:50.645863 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.645837 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:50.695119 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.695086 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-trusted-ca-bundle\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.695308 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.695134 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-service-ca\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.695308 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.695173 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-console-oauth-config\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.695308 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.695210 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-console-serving-cert\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.695308 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.695231 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-console-config\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.695308 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.695305 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6w7p\" (UniqueName: \"kubernetes.io/projected/eee86427-85f4-47aa-816e-754a5fd32f2d-kube-api-access-h6w7p\") pod \"monitoring-plugin-5f68df7f58-4m48s\" (UID: \"eee86427-85f4-47aa-816e-754a5fd32f2d\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-4m48s" Jul 13 18:59:50.695567 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.695408 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eee86427-85f4-47aa-816e-754a5fd32f2d-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-4m48s\" (UID: \"eee86427-85f4-47aa-816e-754a5fd32f2d\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-4m48s" Jul 13 18:59:50.695567 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.695446 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nvcz\" (UniqueName: \"kubernetes.io/projected/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-kube-api-access-5nvcz\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.695567 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.695491 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-oauth-serving-cert\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.796444 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.796371 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-trusted-ca-bundle\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.796444 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.796421 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-service-ca\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.796659 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.796458 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-console-oauth-config\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.796659 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.796488 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-console-serving-cert\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.796659 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.796513 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-console-config\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.796659 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.796562 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6w7p\" (UniqueName: \"kubernetes.io/projected/eee86427-85f4-47aa-816e-754a5fd32f2d-kube-api-access-h6w7p\") pod \"monitoring-plugin-5f68df7f58-4m48s\" (UID: \"eee86427-85f4-47aa-816e-754a5fd32f2d\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-4m48s" Jul 13 18:59:50.796659 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.796623 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eee86427-85f4-47aa-816e-754a5fd32f2d-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-4m48s\" (UID: \"eee86427-85f4-47aa-816e-754a5fd32f2d\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-4m48s" Jul 13 18:59:50.796659 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.796650 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nvcz\" (UniqueName: \"kubernetes.io/projected/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-kube-api-access-5nvcz\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.797007 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.796693 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-oauth-serving-cert\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.797467 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.797441 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-trusted-ca-bundle\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.797732 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.797700 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-service-ca\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.797853 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.797731 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-console-config\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.797917 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.797859 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-oauth-serving-cert\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.799692 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.799668 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-console-oauth-config\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.799819 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.799757 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-console-serving-cert\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.800061 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.800036 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eee86427-85f4-47aa-816e-754a5fd32f2d-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-4m48s\" (UID: \"eee86427-85f4-47aa-816e-754a5fd32f2d\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-4m48s" Jul 13 18:59:50.804681 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.804533 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nvcz\" (UniqueName: \"kubernetes.io/projected/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-kube-api-access-5nvcz\") pod \"console-7755b5d74f-gj8g9\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:50.804681 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.804586 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:50.804977 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.804948 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6w7p\" (UniqueName: \"kubernetes.io/projected/eee86427-85f4-47aa-816e-754a5fd32f2d-kube-api-access-h6w7p\") pod \"monitoring-plugin-5f68df7f58-4m48s\" (UID: \"eee86427-85f4-47aa-816e-754a5fd32f2d\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-4m48s" Jul 13 18:59:50.882642 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.882597 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-4m48s" Jul 13 18:59:50.925987 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.925950 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:50.930005 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:50.929979 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 18:59:51.017656 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.017608 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:51.018816 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.018772 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-cdcc49956-wnbmr"] Jul 13 18:59:51.029371 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.029346 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.031756 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.031728 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 13 18:59:51.031927 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.031764 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 13 18:59:51.032095 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.032068 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 13 18:59:51.032200 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.032103 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 13 18:59:51.032260 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.032075 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-jf6mn\"" Jul 13 18:59:51.032482 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.032460 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 13 18:59:51.041465 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.041102 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-cdcc49956-wnbmr"] Jul 13 18:59:51.044393 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.044368 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 13 18:59:51.100083 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.099997 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f739872a-0869-4559-9a2a-8690e8c630a6-serving-certs-ca-bundle\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.100083 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.100035 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f739872a-0869-4559-9a2a-8690e8c630a6-secret-telemeter-client\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.100083 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.100055 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f739872a-0869-4559-9a2a-8690e8c630a6-federate-client-tls\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.100318 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.100221 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f739872a-0869-4559-9a2a-8690e8c630a6-metrics-client-ca\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.100318 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.100260 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqttr\" (UniqueName: \"kubernetes.io/projected/f739872a-0869-4559-9a2a-8690e8c630a6-kube-api-access-tqttr\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.100318 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.100285 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f739872a-0869-4559-9a2a-8690e8c630a6-telemeter-client-tls\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.100468 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.100374 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f739872a-0869-4559-9a2a-8690e8c630a6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.100468 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.100419 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f739872a-0869-4559-9a2a-8690e8c630a6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.201437 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.201402 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f739872a-0869-4559-9a2a-8690e8c630a6-metrics-client-ca\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.201618 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.201442 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqttr\" (UniqueName: \"kubernetes.io/projected/f739872a-0869-4559-9a2a-8690e8c630a6-kube-api-access-tqttr\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.201618 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.201476 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f739872a-0869-4559-9a2a-8690e8c630a6-telemeter-client-tls\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.201618 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.201515 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f739872a-0869-4559-9a2a-8690e8c630a6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.201618 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.201543 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f739872a-0869-4559-9a2a-8690e8c630a6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.201618 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.201589 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f739872a-0869-4559-9a2a-8690e8c630a6-serving-certs-ca-bundle\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.201618 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.201611 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f739872a-0869-4559-9a2a-8690e8c630a6-secret-telemeter-client\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.202107 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.201636 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f739872a-0869-4559-9a2a-8690e8c630a6-federate-client-tls\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.202381 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.202327 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f739872a-0869-4559-9a2a-8690e8c630a6-metrics-client-ca\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.202577 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.202499 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f739872a-0869-4559-9a2a-8690e8c630a6-serving-certs-ca-bundle\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.202682 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.202646 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f739872a-0869-4559-9a2a-8690e8c630a6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.204562 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.204532 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f739872a-0869-4559-9a2a-8690e8c630a6-telemeter-client-tls\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.204724 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.204703 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f739872a-0869-4559-9a2a-8690e8c630a6-secret-telemeter-client\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.204925 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.204903 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f739872a-0869-4559-9a2a-8690e8c630a6-federate-client-tls\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.205002 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.204934 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f739872a-0869-4559-9a2a-8690e8c630a6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.209936 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.209915 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqttr\" (UniqueName: \"kubernetes.io/projected/f739872a-0869-4559-9a2a-8690e8c630a6-kube-api-access-tqttr\") pod \"telemeter-client-cdcc49956-wnbmr\" (UID: \"f739872a-0869-4559-9a2a-8690e8c630a6\") " pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.346852 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.346817 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" Jul 13 18:59:51.431693 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.431655 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:51.489516 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:51.489482 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:52.033833 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.033594 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 18:59:52.033833 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.033708 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:52.063613 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.063532 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.064750 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.064728 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 18:59:52.068662 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.067518 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 13 18:59:52.068662 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.067760 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 13 18:59:52.068662 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.067820 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 13 18:59:52.068662 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.068065 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 13 18:59:52.068662 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.068097 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 13 18:59:52.068662 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.068317 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 13 18:59:52.068662 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.068347 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 13 18:59:52.068662 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.068466 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 13 18:59:52.068662 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.068472 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 13 18:59:52.068662 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.068598 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 13 18:59:52.069684 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.069338 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 13 18:59:52.069684 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.069554 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cjm6n0v6b8e81\"" Jul 13 18:59:52.070065 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.069872 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zkhw5\"" Jul 13 18:59:52.070065 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.069904 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 13 18:59:52.072970 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.072941 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 13 18:59:52.077680 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.076963 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 13 18:59:52.210998 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.210816 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.210998 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.210886 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.210998 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.210938 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.210998 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.210979 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.211677 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.211011 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.211677 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.211033 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.211677 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.211060 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.211677 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.211085 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-config\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.211677 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.211110 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.213106 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.213076 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.213283 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.213120 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e1e1de81-a666-4de5-976f-5ac4a6ec9235-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.213283 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.213158 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khxqd\" (UniqueName: \"kubernetes.io/projected/e1e1de81-a666-4de5-976f-5ac4a6ec9235-kube-api-access-khxqd\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.213283 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.213256 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e1e1de81-a666-4de5-976f-5ac4a6ec9235-config-out\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.213458 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.213350 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.213458 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.213383 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.213458 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.213416 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.213458 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.213439 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-web-config\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.213642 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.213473 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.213642 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.213497 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.213642 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.213526 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.314756 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.314711 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.314960 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.314856 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-config\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.314960 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.314900 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.314960 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.314928 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.314960 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.314947 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e1e1de81-a666-4de5-976f-5ac4a6ec9235-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.315176 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.314968 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khxqd\" (UniqueName: \"kubernetes.io/projected/e1e1de81-a666-4de5-976f-5ac4a6ec9235-kube-api-access-khxqd\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.315176 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.315004 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e1e1de81-a666-4de5-976f-5ac4a6ec9235-config-out\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.315176 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.315052 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.315176 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.315069 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.315176 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.315097 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.315176 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.315114 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-web-config\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.315450 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.315177 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.315450 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.315202 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.315450 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.315228 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.315450 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.315294 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.315450 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.315317 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.315450 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.315376 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.315732 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.315621 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.316388 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.316359 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.316510 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.316360 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.317207 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.317178 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.318670 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.317977 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-config\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.318670 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.318284 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.318670 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.318523 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.318670 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.318634 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.318962 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.318689 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.318962 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.318704 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.318962 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.318734 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.318962 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.318763 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.320386 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.319263 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.320386 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.320067 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-web-config\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.320386 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.320192 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.322751 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.322672 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.323363 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.323331 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.323611 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.323593 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e1e1de81-a666-4de5-976f-5ac4a6ec9235-config-out\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.323671 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.323648 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.323833 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.323773 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.324086 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.324060 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e1e1de81-a666-4de5-976f-5ac4a6ec9235-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.325750 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.325725 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.327479 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.327455 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khxqd\" (UniqueName: \"kubernetes.io/projected/e1e1de81-a666-4de5-976f-5ac4a6ec9235-kube-api-access-khxqd\") pod \"prometheus-k8s-0\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.381681 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.381619 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 18:59:52.836980 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.836944 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:52.900842 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:52.900799 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:54.299247 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:54.299210 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:55.107246 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.106325 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:55.342969 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.341877 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-99zld"] Jul 13 18:59:55.361598 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.355303 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-4m48s"] Jul 13 18:59:55.361598 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.357051 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f56c8fc58-wdt69"] Jul 13 18:59:55.377055 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:59:55.377021 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c8bb84e_6492_4673_a8e1_d20f38615d40.slice/crio-33c3e12c0a52430e1d2a3c0db3930977f05696bd10b68d714b1884107471652d WatchSource:0}: Error finding container 33c3e12c0a52430e1d2a3c0db3930977f05696bd10b68d714b1884107471652d: Status 404 returned error can't find the container with id 33c3e12c0a52430e1d2a3c0db3930977f05696bd10b68d714b1884107471652d Jul 13 18:59:55.387616 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.387163 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7755b5d74f-gj8g9"] Jul 13 18:59:55.421266 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.416556 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b7c576458-z48dk"] Jul 13 18:59:55.436830 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.424733 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-cdcc49956-wnbmr"] Jul 13 18:59:55.440151 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:59:55.439693 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf739872a_0869_4559_9a2a_8690e8c630a6.slice/crio-bf685a7de39611eb3c93d1c68fac9473cbe236782b96e8ff55eb0c2e5c1d5df8 WatchSource:0}: Error finding container bf685a7de39611eb3c93d1c68fac9473cbe236782b96e8ff55eb0c2e5c1d5df8: Status 404 returned error can't find the container with id bf685a7de39611eb3c93d1c68fac9473cbe236782b96e8ff55eb0c2e5c1d5df8 Jul 13 18:59:55.441260 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.440627 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79664f77cb-c64s2"] Jul 13 18:59:55.441416 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.441357 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cbbbf849d-gwwnr" event={"ID":"99241bde-9e5c-428e-9714-bf8b500410c2","Type":"ContainerStarted","Data":"8a262ec3e0934790e798b9f969cc07578e4e304bb453f4ee6cd762a3ba6bf4ae"} Jul 13 18:59:55.442278 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:59:55.442257 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod980c5a95_d03c_4018_9432_2db17d31c30b.slice/crio-ffec22a92ebe524e6f6962e733a6524bab1dfa40e348ed9d9da7fe92d39a4805 WatchSource:0}: Error finding container ffec22a92ebe524e6f6962e733a6524bab1dfa40e348ed9d9da7fe92d39a4805: Status 404 returned error can't find the container with id ffec22a92ebe524e6f6962e733a6524bab1dfa40e348ed9d9da7fe92d39a4805 Jul 13 18:59:55.443217 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.443183 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5ftlh" event={"ID":"e8bde243-4f0c-4fd7-a18c-b5900c1609e8","Type":"ContainerStarted","Data":"1bd7502970d9f52b83d5a6c8283d3ba51ec3c3a718f48bec7ca4cb6103540db8"} Jul 13 18:59:55.445349 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:59:55.445097 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaa1640a_5f83_4a1a_aa77_fbcbb321acb6.slice/crio-b84a8aea92c57df7e8366e58e46cd5f21cae3e923c08a675cee143d4960b070d WatchSource:0}: Error finding container b84a8aea92c57df7e8366e58e46cd5f21cae3e923c08a675cee143d4960b070d: Status 404 returned error can't find the container with id b84a8aea92c57df7e8366e58e46cd5f21cae3e923c08a675cee143d4960b070d Jul 13 18:59:55.446859 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.446807 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f4ffp" event={"ID":"b78c80c9-b3b8-40a6-a842-da04dbeddf03","Type":"ContainerStarted","Data":"754e0faea8bbc97ab587e08e4a5a0d4b3b9c25ca9c551994ff4b253ab39a9131"} Jul 13 18:59:55.448369 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.448315 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8kqlr" event={"ID":"4ca24210-be70-483b-9893-eece94d2756b","Type":"ContainerStarted","Data":"17a2a6391d1a06f9d6ed5de0d889b62ce88497b4c4dcfd5e0e48aeb97ebbc996"} Jul 13 18:59:55.451311 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.451247 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 18:59:55.451619 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.451594 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7755b5d74f-gj8g9" event={"ID":"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86","Type":"ContainerStarted","Data":"813193f460d29da77cf46aacece3aeceddaff4a3da95f6e82e9dcc9bcc230761"} Jul 13 18:59:55.453473 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.453396 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" event={"ID":"3f3a9e5d-3247-474e-85fb-1026a2779ee1","Type":"ContainerStarted","Data":"36a0dee2d87f9e9f13cee681ce5f0bdc67e4b789e2fdb9104775c8c42586bd4d"} Jul 13 18:59:55.455714 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.455107 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f56c8fc58-wdt69" event={"ID":"8c8bb84e-6492-4673-a8e1-d20f38615d40","Type":"ContainerStarted","Data":"33c3e12c0a52430e1d2a3c0db3930977f05696bd10b68d714b1884107471652d"} Jul 13 18:59:55.457465 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.457441 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-4m48s" event={"ID":"eee86427-85f4-47aa-816e-754a5fd32f2d","Type":"ContainerStarted","Data":"dfc768306129087c9e72cbddc8d53c8bc59ca18935a996f0930a2b44ace00169"} Jul 13 18:59:55.466772 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.465804 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cbbbf849d-gwwnr" podStartSLOduration=1.57778598 podStartE2EDuration="16.465766561s" totalImagesPullingTime="14.887980581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:55.465112575 +0000 UTC m=+80.941566773" watchObservedRunningTime="2026-07-13 18:59:55.465766561 +0000 UTC m=+80.942220739" Jul 13 18:59:55.481573 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.481520 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5ftlh" podStartSLOduration=67.478835389 podStartE2EDuration="1m21.48149965s" totalImagesPullingTime="14.002664261s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:55.480446918 +0000 UTC m=+80.956901116" watchObservedRunningTime="2026-07-13 18:59:55.48149965 +0000 UTC m=+80.957953826" Jul 13 18:59:55.496533 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.496484 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-f4ffp" podStartSLOduration=67.914703785 podStartE2EDuration="1m21.496466899s" totalImagesPullingTime="13.581763114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:55.495818635 +0000 UTC m=+80.972272810" watchObservedRunningTime="2026-07-13 18:59:55.496466899 +0000 UTC m=+80.972921074" Jul 13 18:59:55.523621 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.523541 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr"] Jul 13 18:59:55.529959 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:59:55.529338 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62c27f1a_06ed_42aa_a152_e96e9fc07478.slice/crio-416d9867c33fba6b7e31194588dfa0cd98fc360d5305b9ebaf4c7904a95fe50c WatchSource:0}: Error finding container 416d9867c33fba6b7e31194588dfa0cd98fc360d5305b9ebaf4c7904a95fe50c: Status 404 returned error can't find the container with id 416d9867c33fba6b7e31194588dfa0cd98fc360d5305b9ebaf4c7904a95fe50c Jul 13 18:59:55.670156 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.670106 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 18:59:55.672950 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:59:55.672847 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda720e726_5e5b_44c4_83c7_06f5c5aa4554.slice/crio-59cf82e73228a021418f21ee33a5781782a32f489351c8fb3f078796c40a6aab WatchSource:0}: Error finding container 59cf82e73228a021418f21ee33a5781782a32f489351c8fb3f078796c40a6aab: Status 404 returned error can't find the container with id 59cf82e73228a021418f21ee33a5781782a32f489351c8fb3f078796c40a6aab Jul 13 18:59:55.675680 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.675653 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 18:59:55.678394 ip-10-0-128-30 kubenswrapper[2574]: W0713 18:59:55.678370 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1e1de81_a666_4de5_976f_5ac4a6ec9235.slice/crio-18c2f4c1d365bcff205807a749d7fe0d8db8f4d95e290ea782a6110a47e0f50f WatchSource:0}: Error finding container 18c2f4c1d365bcff205807a749d7fe0d8db8f4d95e290ea782a6110a47e0f50f: Status 404 returned error can't find the container with id 18c2f4c1d365bcff205807a749d7fe0d8db8f4d95e290ea782a6110a47e0f50f Jul 13 18:59:55.933694 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:55.933606 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:56.015359 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.015277 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:56.143724 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.143652 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:56.223466 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.223382 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 18:59:56.471815 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.471425 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7755b5d74f-gj8g9" event={"ID":"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86","Type":"ContainerStarted","Data":"9e6f2ce917e4ec8025e661ae871707184a9708dd7ad8fd25a247290afc9d5b80"} Jul 13 18:59:56.475468 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.475006 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" event={"ID":"f739872a-0869-4559-9a2a-8690e8c630a6","Type":"ContainerStarted","Data":"bf685a7de39611eb3c93d1c68fac9473cbe236782b96e8ff55eb0c2e5c1d5df8"} Jul 13 18:59:56.487113 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.487077 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a720e726-5e5b-44c4-83c7-06f5c5aa4554","Type":"ContainerStarted","Data":"59cf82e73228a021418f21ee33a5781782a32f489351c8fb3f078796c40a6aab"} Jul 13 18:59:56.491577 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.491440 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" event={"ID":"62c27f1a-06ed-42aa-a152-e96e9fc07478","Type":"ContainerStarted","Data":"8135f4e7c84ef92533a1e67fcc5ce69027c7ef959ddd94c709055330c6a2dc4a"} Jul 13 18:59:56.491577 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.491501 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" event={"ID":"62c27f1a-06ed-42aa-a152-e96e9fc07478","Type":"ContainerStarted","Data":"f816b07e73c891d38c45a1de12870ca424f133df26a6371c72821a5f3ed90867"} Jul 13 18:59:56.491577 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.491531 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" event={"ID":"62c27f1a-06ed-42aa-a152-e96e9fc07478","Type":"ContainerStarted","Data":"416d9867c33fba6b7e31194588dfa0cd98fc360d5305b9ebaf4c7904a95fe50c"} Jul 13 18:59:56.494083 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.494030 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7755b5d74f-gj8g9" podStartSLOduration=6.494013107 podStartE2EDuration="6.494013107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:56.493559421 +0000 UTC m=+81.970013608" watchObservedRunningTime="2026-07-13 18:59:56.494013107 +0000 UTC m=+81.970467283" Jul 13 18:59:56.497390 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.497358 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f56c8fc58-wdt69" event={"ID":"8c8bb84e-6492-4673-a8e1-d20f38615d40","Type":"ContainerStarted","Data":"8056f0dc611607b48b91238af8c072eda16332697531f04ed9a0d1ae36f4d22a"} Jul 13 18:59:56.498801 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.498756 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" event={"ID":"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6","Type":"ContainerStarted","Data":"b84a8aea92c57df7e8366e58e46cd5f21cae3e923c08a675cee143d4960b070d"} Jul 13 18:59:56.503188 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.503161 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" event={"ID":"980c5a95-d03c-4018-9432-2db17d31c30b","Type":"ContainerStarted","Data":"ffec22a92ebe524e6f6962e733a6524bab1dfa40e348ed9d9da7fe92d39a4805"} Jul 13 18:59:56.505631 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.505606 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-xpkqs" event={"ID":"3decbc78-37dd-41de-840a-eb02d2507656","Type":"ContainerStarted","Data":"a21072312d91533dcd2efa32186ef056bae9d548cd809232525d23855c707456"} Jul 13 18:59:56.506464 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.506409 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-xpkqs" Jul 13 18:59:56.509863 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.509818 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1e1de81-a666-4de5-976f-5ac4a6ec9235","Type":"ContainerStarted","Data":"18c2f4c1d365bcff205807a749d7fe0d8db8f4d95e290ea782a6110a47e0f50f"} Jul 13 18:59:56.517108 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.517059 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-xpkqs" Jul 13 18:59:56.519246 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.519199 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f56c8fc58-wdt69" podStartSLOduration=12.519186332 podStartE2EDuration="12.519186332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:56.518126472 +0000 UTC m=+81.994580678" watchObservedRunningTime="2026-07-13 18:59:56.519186332 +0000 UTC m=+81.995640506" Jul 13 18:59:56.534440 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:56.534336 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-xpkqs" podStartSLOduration=1.966957078 podStartE2EDuration="21.534322253s" totalImagesPullingTime="19.567365175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:56.533917123 +0000 UTC m=+82.010371297" watchObservedRunningTime="2026-07-13 18:59:56.534322253 +0000 UTC m=+82.010776427" Jul 13 18:59:59.887720 ip-10-0-128-30 kubenswrapper[2574]: I0713 18:59:59.887682 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 19:00:00.440536 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:00.440506 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:00.541134 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:00.541098 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:00.930381 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:00.930338 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 19:00:00.930381 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:00.930406 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 19:00:00.936418 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:00.936389 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 19:00:01.546962 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:01.546928 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 19:00:01.611704 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:01.611576 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f56c8fc58-wdt69"] Jul 13 19:00:03.391146 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.391103 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:03.552717 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.552463 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" event={"ID":"eaa1640a-5f83-4a1a-aa77-fbcbb321acb6","Type":"ContainerStarted","Data":"74004596fdcbcd04eb33040019f0f4aa0a5b0df07f750cdcf7d6513550c46956"} Jul 13 19:00:03.554951 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.554672 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" event={"ID":"980c5a95-d03c-4018-9432-2db17d31c30b","Type":"ContainerStarted","Data":"edc3fd8b64b1cdb0b1d86de496be839ec344ecbf9b23ff811f13d8accde2c072"} Jul 13 19:00:03.554951 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.554743 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" event={"ID":"980c5a95-d03c-4018-9432-2db17d31c30b","Type":"ContainerStarted","Data":"9ad4f90e55d4294851bcf4d66736752e1ec08bf2b52d57b8c8a6eadc0a427878"} Jul 13 19:00:03.557704 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.557613 2574 generic.go:356] "Generic (PLEG): container finished" podID="4ca24210-be70-483b-9893-eece94d2756b" containerID="c56a0d0da307245d8bdf46dc59a6188f8a6020b00e0179b7223612666778b46d" exitCode=0 Jul 13 19:00:03.557835 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.557707 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8kqlr" event={"ID":"4ca24210-be70-483b-9893-eece94d2756b","Type":"ContainerDied","Data":"c56a0d0da307245d8bdf46dc59a6188f8a6020b00e0179b7223612666778b46d"} Jul 13 19:00:03.561745 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.561719 2574 generic.go:356] "Generic (PLEG): container finished" podID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerID="e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26" exitCode=0 Jul 13 19:00:03.561883 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.561824 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1e1de81-a666-4de5-976f-5ac4a6ec9235","Type":"ContainerDied","Data":"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26"} Jul 13 19:00:03.564676 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.564465 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" event={"ID":"f739872a-0869-4559-9a2a-8690e8c630a6","Type":"ContainerStarted","Data":"ec75e2eb3c5d20d04bdeeb520679abe715ebf59c6ad88c700b1df61d1ffca5bf"} Jul 13 19:00:03.564676 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.564532 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" event={"ID":"f739872a-0869-4559-9a2a-8690e8c630a6","Type":"ContainerStarted","Data":"c94c531eb7faf58672052199e278045965d8a8c47ebbac02bd2b4228b1df5618"} Jul 13 19:00:03.566972 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.566837 2574 generic.go:356] "Generic (PLEG): container finished" podID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerID="b1957ff012aa4425ce169a811355aaf37d33128ba982e8b6b4f833e8c6697fd8" exitCode=0 Jul 13 19:00:03.566972 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.566930 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a720e726-5e5b-44c4-83c7-06f5c5aa4554","Type":"ContainerDied","Data":"b1957ff012aa4425ce169a811355aaf37d33128ba982e8b6b4f833e8c6697fd8"} Jul 13 19:00:03.573845 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.570413 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-4m48s" event={"ID":"eee86427-85f4-47aa-816e-754a5fd32f2d","Type":"ContainerStarted","Data":"4f45bc3c033a861e07835dcfaf51cb862234493aed346c24d6e470329c41d5cb"} Jul 13 19:00:03.573845 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.571171 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-4m48s" Jul 13 19:00:03.582391 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.582357 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" event={"ID":"3f3a9e5d-3247-474e-85fb-1026a2779ee1","Type":"ContainerStarted","Data":"4ad1fbfcaafaf76db1abfb5bac6ee545e1cbd9ff54f7c6f50102e5cdfa5ecfd7"} Jul 13 19:00:03.582518 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.582464 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" event={"ID":"3f3a9e5d-3247-474e-85fb-1026a2779ee1","Type":"ContainerStarted","Data":"6f8eb2a1559bf2a7821f2bac680e232ca978620efc61ecdef436106268de0ae6"} Jul 13 19:00:03.582518 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.582501 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" event={"ID":"3f3a9e5d-3247-474e-85fb-1026a2779ee1","Type":"ContainerStarted","Data":"4874de6e6012f1f00e8560fd811e44fce0e6c553a24e4a09d9b8f35804ecdaa4"} Jul 13 19:00:03.585463 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.585432 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" event={"ID":"62c27f1a-06ed-42aa-a152-e96e9fc07478","Type":"ContainerStarted","Data":"573a2f8ebc6fbe62bbcc382f053c899d2fa7fdf4a8ff133c549055db2789a700"} Jul 13 19:00:03.587451 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.586985 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" podStartSLOduration=6.3886729540000005 podStartE2EDuration="13.586961153s" totalImagesPullingTime="7.198288199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:00:03.571990032 +0000 UTC m=+89.048444233" watchObservedRunningTime="2026-07-13 19:00:03.586961153 +0000 UTC m=+89.063415333" Jul 13 19:00:03.591851 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.591829 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-4m48s" Jul 13 19:00:03.676018 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.675946 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-4m48s" podStartSLOduration=6.787325679 podStartE2EDuration="13.675922108s" totalImagesPullingTime="6.888596429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:00:03.673909552 +0000 UTC m=+89.150363755" watchObservedRunningTime="2026-07-13 19:00:03.675922108 +0000 UTC m=+89.152376283" Jul 13 19:00:03.695559 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.695493 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-b2htr" podStartSLOduration=11.37839044 podStartE2EDuration="18.695470721s" totalImagesPullingTime="7.317080281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:00:03.693893237 +0000 UTC m=+89.170347412" watchObservedRunningTime="2026-07-13 19:00:03.695470721 +0000 UTC m=+89.171924896" Jul 13 19:00:03.737203 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:03.735762 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-99zld" podStartSLOduration=11.448746222 podStartE2EDuration="18.735741137s" totalImagesPullingTime="7.286994915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:00:03.73556565 +0000 UTC m=+89.212019826" watchObservedRunningTime="2026-07-13 19:00:03.735741137 +0000 UTC m=+89.212195314" Jul 13 19:00:04.594645 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:04.594594 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" event={"ID":"980c5a95-d03c-4018-9432-2db17d31c30b","Type":"ContainerStarted","Data":"6330a44f03d3fcbc72018035b0d250e1eefe9e761299dea04dd328293837f7f3"} Jul 13 19:00:04.597283 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:04.597243 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8kqlr" event={"ID":"4ca24210-be70-483b-9893-eece94d2756b","Type":"ContainerStarted","Data":"af4b313e0d0162766bd9853ad142a90210c30add256b9751cfb508fb8f26c2cf"} Jul 13 19:00:04.597437 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:04.597319 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8kqlr" event={"ID":"4ca24210-be70-483b-9893-eece94d2756b","Type":"ContainerStarted","Data":"54817b6793d45bef478701645c45f8f12b0ac810f2987cdfd5efdaa5506ef93d"} Jul 13 19:00:04.600044 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:04.600012 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" event={"ID":"f739872a-0869-4559-9a2a-8690e8c630a6","Type":"ContainerStarted","Data":"fe930d7177bf75de6c01486cf127f18b64df4d5591630e68b1499496770cb526"} Jul 13 19:00:04.617502 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:04.617419 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8kqlr" podStartSLOduration=11.918716707 podStartE2EDuration="19.617398642s" totalImagesPullingTime="7.527154982s" totalInitContainerRuntime="171.526953ms" isStatefulPod=true podCreationTimestamp="2026-07-13 18:59:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:00:04.616597881 +0000 UTC m=+90.093052056" watchObservedRunningTime="2026-07-13 19:00:04.617398642 +0000 UTC m=+90.093852817" Jul 13 19:00:04.643117 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:04.643053 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-cdcc49956-wnbmr" podStartSLOduration=7.406327063 podStartE2EDuration="14.643030489s" totalImagesPullingTime="7.236703426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:00:04.639496907 +0000 UTC m=+90.115951108" watchObservedRunningTime="2026-07-13 19:00:04.643030489 +0000 UTC m=+90.119484664" Jul 13 19:00:04.689266 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:04.689222 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 19:00:04.802715 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:04.802682 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:04.969436 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:04.969331 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:06.613381 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:06.613348 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a720e726-5e5b-44c4-83c7-06f5c5aa4554","Type":"ContainerStarted","Data":"ca9b4360b067d1259bae27ea2e1c0fc673f9a7c8d0e525a61e0429f88ce9ad16"} Jul 13 19:00:06.613817 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:06.613417 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a720e726-5e5b-44c4-83c7-06f5c5aa4554","Type":"ContainerStarted","Data":"eadec454641cff51eda8d6e7652aa0bec1b58bf833a73d80461f658d60ca527f"} Jul 13 19:00:06.613817 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:06.613443 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a720e726-5e5b-44c4-83c7-06f5c5aa4554","Type":"ContainerStarted","Data":"3946ff2516abf18b1eb04e95fa8dcad151319cbcb7a2bcb112e47608f54f6b8b"} Jul 13 19:00:06.613817 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:06.613467 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a720e726-5e5b-44c4-83c7-06f5c5aa4554","Type":"ContainerStarted","Data":"e94a9a44b44aacd332cc2cfa3d3f662f54acee98d98bd90eaf3772757208fb92"} Jul 13 19:00:06.613817 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:06.613492 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a720e726-5e5b-44c4-83c7-06f5c5aa4554","Type":"ContainerStarted","Data":"148a19b74445adc83dbdda78d3f94057f14771ede18849ec250d837aab58a066"} Jul 13 19:00:07.151520 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:07.150992 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:07.622222 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:07.622176 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" event={"ID":"980c5a95-d03c-4018-9432-2db17d31c30b","Type":"ContainerStarted","Data":"930b9b4bdc228caf40cd4eedd77194e022b93f5fe56a41c02e1c751e1b5d3bb0"} Jul 13 19:00:07.622707 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:07.622248 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" event={"ID":"980c5a95-d03c-4018-9432-2db17d31c30b","Type":"ContainerStarted","Data":"56a88e02c714dd8ad7cec3d716e91ba713ea14e6487a6c5e11f78c94b1defb6f"} Jul 13 19:00:07.622707 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:07.622280 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" event={"ID":"980c5a95-d03c-4018-9432-2db17d31c30b","Type":"ContainerStarted","Data":"20c1a3695705387a5e3f2c50c84ace92cee2c17adafbdf93f682e3fd18ba1a9e"} Jul 13 19:00:07.622707 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:07.622349 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 19:00:07.646901 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:07.646846 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" podStartSLOduration=9.258415764 podStartE2EDuration="19.646828139s" totalImagesPullingTime="10.388412375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:00:07.644589299 +0000 UTC m=+93.121043474" watchObservedRunningTime="2026-07-13 19:00:07.646828139 +0000 UTC m=+93.123282313" Jul 13 19:00:08.526670 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:08.526632 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:08.632900 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:08.632868 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-b7c576458-z48dk" Jul 13 19:00:09.634940 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:09.634905 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1e1de81-a666-4de5-976f-5ac4a6ec9235","Type":"ContainerStarted","Data":"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179"} Jul 13 19:00:09.635233 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:09.635030 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1e1de81-a666-4de5-976f-5ac4a6ec9235","Type":"ContainerStarted","Data":"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec"} Jul 13 19:00:09.635233 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:09.635076 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1e1de81-a666-4de5-976f-5ac4a6ec9235","Type":"ContainerStarted","Data":"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd"} Jul 13 19:00:09.635233 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:09.635115 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1e1de81-a666-4de5-976f-5ac4a6ec9235","Type":"ContainerStarted","Data":"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1"} Jul 13 19:00:09.637994 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:09.637966 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a720e726-5e5b-44c4-83c7-06f5c5aa4554","Type":"ContainerStarted","Data":"9bb595ed2d78e1db3fe1727645439052e79b5de625402e183a51982b5da7f664"} Jul 13 19:00:09.669114 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:09.669066 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=12.15098844 podStartE2EDuration="23.669045259s" totalImagesPullingTime="11.504160848s" totalInitContainerRuntime="13.895971ms" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:00:09.666022413 +0000 UTC m=+95.142476587" watchObservedRunningTime="2026-07-13 19:00:09.669045259 +0000 UTC m=+95.145499434" Jul 13 19:00:10.541969 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:10.541928 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 19:00:10.542207 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:10.541989 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 19:00:10.649408 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:10.649364 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1e1de81-a666-4de5-976f-5ac4a6ec9235","Type":"ContainerStarted","Data":"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4"} Jul 13 19:00:10.649878 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:10.649450 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1e1de81-a666-4de5-976f-5ac4a6ec9235","Type":"ContainerStarted","Data":"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9"} Jul 13 19:00:10.691879 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:10.691824 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.156027166 podStartE2EDuration="18.691806688s" totalImagesPullingTime="12.466176827s" totalInitContainerRuntime="69.602695ms" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:00:10.688592988 +0000 UTC m=+96.165047192" watchObservedRunningTime="2026-07-13 19:00:10.691806688 +0000 UTC m=+96.168260852" Jul 13 19:00:11.018570 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:11.018495 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:11.092844 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:11.092813 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:12.382309 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:12.382268 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:00:16.261390 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:16.261356 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7755b5d74f-gj8g9"] Jul 13 19:00:16.417305 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:16.417259 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:16.526056 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:16.525980 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:20.474551 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:20.474489 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5cbbbf849d-gwwnr" podUID="99241bde-9e5c-428e-9714-bf8b500410c2" containerName="console" containerID="cri-o://8a262ec3e0934790e798b9f969cc07578e4e304bb453f4ee6cd762a3ba6bf4ae" gracePeriod=15 Jul 13 19:00:20.696413 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:20.696381 2574 generic.go:356] "Generic (PLEG): container finished" podID="99241bde-9e5c-428e-9714-bf8b500410c2" containerID="8a262ec3e0934790e798b9f969cc07578e4e304bb453f4ee6cd762a3ba6bf4ae" exitCode=0 Jul 13 19:00:20.696563 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:20.696454 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cbbbf849d-gwwnr" event={"ID":"99241bde-9e5c-428e-9714-bf8b500410c2","Type":"ContainerDied","Data":"8a262ec3e0934790e798b9f969cc07578e4e304bb453f4ee6cd762a3ba6bf4ae"} Jul 13 19:00:20.768628 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:20.768602 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 19:00:20.913900 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:20.913871 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:20.934010 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:20.933985 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99241bde-9e5c-428e-9714-bf8b500410c2-console-config\") pod \"99241bde-9e5c-428e-9714-bf8b500410c2\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " Jul 13 19:00:20.934118 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:20.934040 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99241bde-9e5c-428e-9714-bf8b500410c2-console-serving-cert\") pod \"99241bde-9e5c-428e-9714-bf8b500410c2\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " Jul 13 19:00:20.934118 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:20.934075 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99241bde-9e5c-428e-9714-bf8b500410c2-console-oauth-config\") pod \"99241bde-9e5c-428e-9714-bf8b500410c2\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " Jul 13 19:00:20.934118 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:20.934111 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99241bde-9e5c-428e-9714-bf8b500410c2-oauth-serving-cert\") pod \"99241bde-9e5c-428e-9714-bf8b500410c2\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " Jul 13 19:00:20.934252 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:20.934133 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99241bde-9e5c-428e-9714-bf8b500410c2-service-ca\") pod \"99241bde-9e5c-428e-9714-bf8b500410c2\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " Jul 13 19:00:20.934252 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:20.934211 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fl58j\" (UniqueName: \"kubernetes.io/projected/99241bde-9e5c-428e-9714-bf8b500410c2-kube-api-access-fl58j\") pod \"99241bde-9e5c-428e-9714-bf8b500410c2\" (UID: \"99241bde-9e5c-428e-9714-bf8b500410c2\") " Jul 13 19:00:20.934537 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:20.934491 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99241bde-9e5c-428e-9714-bf8b500410c2-console-config" (OuterVolumeSpecName: "console-config") pod "99241bde-9e5c-428e-9714-bf8b500410c2" (UID: "99241bde-9e5c-428e-9714-bf8b500410c2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:00:20.934650 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:20.934546 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99241bde-9e5c-428e-9714-bf8b500410c2-service-ca" (OuterVolumeSpecName: "service-ca") pod "99241bde-9e5c-428e-9714-bf8b500410c2" (UID: "99241bde-9e5c-428e-9714-bf8b500410c2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:00:20.934650 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:20.934553 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99241bde-9e5c-428e-9714-bf8b500410c2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "99241bde-9e5c-428e-9714-bf8b500410c2" (UID: "99241bde-9e5c-428e-9714-bf8b500410c2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:00:20.936424 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:20.936397 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99241bde-9e5c-428e-9714-bf8b500410c2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "99241bde-9e5c-428e-9714-bf8b500410c2" (UID: "99241bde-9e5c-428e-9714-bf8b500410c2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:00:20.936514 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:20.936449 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99241bde-9e5c-428e-9714-bf8b500410c2-kube-api-access-fl58j" (OuterVolumeSpecName: "kube-api-access-fl58j") pod "99241bde-9e5c-428e-9714-bf8b500410c2" (UID: "99241bde-9e5c-428e-9714-bf8b500410c2"). InnerVolumeSpecName "kube-api-access-fl58j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:00:20.936514 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:20.936483 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99241bde-9e5c-428e-9714-bf8b500410c2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "99241bde-9e5c-428e-9714-bf8b500410c2" (UID: "99241bde-9e5c-428e-9714-bf8b500410c2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:00:20.976641 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:20.976600 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:21.035300 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:21.035217 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99241bde-9e5c-428e-9714-bf8b500410c2-console-serving-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:21.035300 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:21.035245 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99241bde-9e5c-428e-9714-bf8b500410c2-console-oauth-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:21.035300 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:21.035256 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99241bde-9e5c-428e-9714-bf8b500410c2-oauth-serving-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:21.035300 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:21.035267 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99241bde-9e5c-428e-9714-bf8b500410c2-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:21.035300 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:21.035275 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fl58j\" (UniqueName: \"kubernetes.io/projected/99241bde-9e5c-428e-9714-bf8b500410c2-kube-api-access-fl58j\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:21.035300 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:21.035284 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99241bde-9e5c-428e-9714-bf8b500410c2-console-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:21.701660 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:21.701631 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cbbbf849d-gwwnr" Jul 13 19:00:21.702066 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:21.701630 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cbbbf849d-gwwnr" event={"ID":"99241bde-9e5c-428e-9714-bf8b500410c2","Type":"ContainerDied","Data":"0f1be58eb3f286a4633b183943d18a5526868f235378332ead510ed1a4a8c978"} Jul 13 19:00:21.702066 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:21.701768 2574 scope.go:117] "RemoveContainer" containerID="8a262ec3e0934790e798b9f969cc07578e4e304bb453f4ee6cd762a3ba6bf4ae" Jul 13 19:00:21.718708 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:21.718681 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cbbbf849d-gwwnr"] Jul 13 19:00:21.722634 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:21.722606 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cbbbf849d-gwwnr"] Jul 13 19:00:23.053986 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:23.053946 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99241bde-9e5c-428e-9714-bf8b500410c2" path="/var/lib/kubelet/pods/99241bde-9e5c-428e-9714-bf8b500410c2/volumes" Jul 13 19:00:23.411403 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:23.411312 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:23.512973 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:23.512939 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:00:26.512167 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:26.512137 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-f4ffp" Jul 13 19:00:26.641694 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:26.641652 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f56c8fc58-wdt69" podUID="8c8bb84e-6492-4673-a8e1-d20f38615d40" containerName="console" containerID="cri-o://8056f0dc611607b48b91238af8c072eda16332697531f04ed9a0d1ae36f4d22a" gracePeriod=15 Jul 13 19:00:26.914536 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:26.914503 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 19:00:27.093091 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.093058 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c8bb84e-6492-4673-a8e1-d20f38615d40-console-oauth-config\") pod \"8c8bb84e-6492-4673-a8e1-d20f38615d40\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " Jul 13 19:00:27.093282 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.093107 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c8bb84e-6492-4673-a8e1-d20f38615d40-console-serving-cert\") pod \"8c8bb84e-6492-4673-a8e1-d20f38615d40\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " Jul 13 19:00:27.093282 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.093144 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkdw2\" (UniqueName: \"kubernetes.io/projected/8c8bb84e-6492-4673-a8e1-d20f38615d40-kube-api-access-wkdw2\") pod \"8c8bb84e-6492-4673-a8e1-d20f38615d40\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " Jul 13 19:00:27.093282 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.093170 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-oauth-serving-cert\") pod \"8c8bb84e-6492-4673-a8e1-d20f38615d40\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " Jul 13 19:00:27.093672 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.093354 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-console-config\") pod \"8c8bb84e-6492-4673-a8e1-d20f38615d40\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " Jul 13 19:00:27.093672 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.093400 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-trusted-ca-bundle\") pod \"8c8bb84e-6492-4673-a8e1-d20f38615d40\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " Jul 13 19:00:27.093672 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.093425 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-service-ca\") pod \"8c8bb84e-6492-4673-a8e1-d20f38615d40\" (UID: \"8c8bb84e-6492-4673-a8e1-d20f38615d40\") " Jul 13 19:00:27.093672 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.093656 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8c8bb84e-6492-4673-a8e1-d20f38615d40" (UID: "8c8bb84e-6492-4673-a8e1-d20f38615d40"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:00:27.093916 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.093726 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-console-config" (OuterVolumeSpecName: "console-config") pod "8c8bb84e-6492-4673-a8e1-d20f38615d40" (UID: "8c8bb84e-6492-4673-a8e1-d20f38615d40"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:00:27.093916 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.093776 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-oauth-serving-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:27.093916 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.093775 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8c8bb84e-6492-4673-a8e1-d20f38615d40" (UID: "8c8bb84e-6492-4673-a8e1-d20f38615d40"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:00:27.094071 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.093976 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-service-ca" (OuterVolumeSpecName: "service-ca") pod "8c8bb84e-6492-4673-a8e1-d20f38615d40" (UID: "8c8bb84e-6492-4673-a8e1-d20f38615d40"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:00:27.095545 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.095523 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c8bb84e-6492-4673-a8e1-d20f38615d40-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8c8bb84e-6492-4673-a8e1-d20f38615d40" (UID: "8c8bb84e-6492-4673-a8e1-d20f38615d40"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:00:27.095888 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.095868 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c8bb84e-6492-4673-a8e1-d20f38615d40-kube-api-access-wkdw2" (OuterVolumeSpecName: "kube-api-access-wkdw2") pod "8c8bb84e-6492-4673-a8e1-d20f38615d40" (UID: "8c8bb84e-6492-4673-a8e1-d20f38615d40"). InnerVolumeSpecName "kube-api-access-wkdw2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:00:27.095973 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.095901 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c8bb84e-6492-4673-a8e1-d20f38615d40-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8c8bb84e-6492-4673-a8e1-d20f38615d40" (UID: "8c8bb84e-6492-4673-a8e1-d20f38615d40"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:00:27.195016 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.194979 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-console-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:27.195016 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.195012 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-trusted-ca-bundle\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:27.195217 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.195027 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8bb84e-6492-4673-a8e1-d20f38615d40-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:27.195217 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.195041 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c8bb84e-6492-4673-a8e1-d20f38615d40-console-oauth-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:27.195217 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.195052 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c8bb84e-6492-4673-a8e1-d20f38615d40-console-serving-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:27.195217 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.195065 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkdw2\" (UniqueName: \"kubernetes.io/projected/8c8bb84e-6492-4673-a8e1-d20f38615d40-kube-api-access-wkdw2\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:27.728605 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.728566 2574 generic.go:356] "Generic (PLEG): container finished" podID="8c8bb84e-6492-4673-a8e1-d20f38615d40" containerID="8056f0dc611607b48b91238af8c072eda16332697531f04ed9a0d1ae36f4d22a" exitCode=0 Jul 13 19:00:27.728605 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.728607 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f56c8fc58-wdt69" Jul 13 19:00:27.729226 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.728649 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f56c8fc58-wdt69" event={"ID":"8c8bb84e-6492-4673-a8e1-d20f38615d40","Type":"ContainerDied","Data":"8056f0dc611607b48b91238af8c072eda16332697531f04ed9a0d1ae36f4d22a"} Jul 13 19:00:27.729226 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.728702 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f56c8fc58-wdt69" event={"ID":"8c8bb84e-6492-4673-a8e1-d20f38615d40","Type":"ContainerDied","Data":"33c3e12c0a52430e1d2a3c0db3930977f05696bd10b68d714b1884107471652d"} Jul 13 19:00:27.729226 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.728723 2574 scope.go:117] "RemoveContainer" containerID="8056f0dc611607b48b91238af8c072eda16332697531f04ed9a0d1ae36f4d22a" Jul 13 19:00:27.739237 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.739218 2574 scope.go:117] "RemoveContainer" containerID="8056f0dc611607b48b91238af8c072eda16332697531f04ed9a0d1ae36f4d22a" Jul 13 19:00:27.739542 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:00:27.739518 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8056f0dc611607b48b91238af8c072eda16332697531f04ed9a0d1ae36f4d22a\": container with ID starting with 8056f0dc611607b48b91238af8c072eda16332697531f04ed9a0d1ae36f4d22a not found: ID does not exist" containerID="8056f0dc611607b48b91238af8c072eda16332697531f04ed9a0d1ae36f4d22a" Jul 13 19:00:27.739591 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.739552 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8056f0dc611607b48b91238af8c072eda16332697531f04ed9a0d1ae36f4d22a"} err="failed to get container status \"8056f0dc611607b48b91238af8c072eda16332697531f04ed9a0d1ae36f4d22a\": rpc error: code = NotFound desc = could not find container \"8056f0dc611607b48b91238af8c072eda16332697531f04ed9a0d1ae36f4d22a\": container with ID starting with 8056f0dc611607b48b91238af8c072eda16332697531f04ed9a0d1ae36f4d22a not found: ID does not exist" Jul 13 19:00:27.751506 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.751486 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f56c8fc58-wdt69"] Jul 13 19:00:27.756464 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:27.756443 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f56c8fc58-wdt69"] Jul 13 19:00:29.053535 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:29.053503 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c8bb84e-6492-4673-a8e1-d20f38615d40" path="/var/lib/kubelet/pods/8c8bb84e-6492-4673-a8e1-d20f38615d40/volumes" Jul 13 19:00:30.547423 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:30.547389 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 19:00:30.551327 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:30.551296 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-79664f77cb-c64s2" Jul 13 19:00:41.282553 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.282507 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7755b5d74f-gj8g9" podUID="3ef99af9-efaa-44ba-a64c-9b43b8ce6f86" containerName="console" containerID="cri-o://9e6f2ce917e4ec8025e661ae871707184a9708dd7ad8fd25a247290afc9d5b80" gracePeriod=15 Jul 13 19:00:41.567150 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.567117 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 19:00:41.622053 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.622011 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-console-oauth-config\") pod \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " Jul 13 19:00:41.622053 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.622055 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-trusted-ca-bundle\") pod \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " Jul 13 19:00:41.622241 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.622080 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-console-config\") pod \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " Jul 13 19:00:41.622241 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.622118 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-console-serving-cert\") pod \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " Jul 13 19:00:41.622241 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.622192 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nvcz\" (UniqueName: \"kubernetes.io/projected/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-kube-api-access-5nvcz\") pod \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " Jul 13 19:00:41.622241 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.622229 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-oauth-serving-cert\") pod \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " Jul 13 19:00:41.622502 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.622259 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-service-ca\") pod \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\" (UID: \"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86\") " Jul 13 19:00:41.622579 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.622541 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-console-config" (OuterVolumeSpecName: "console-config") pod "3ef99af9-efaa-44ba-a64c-9b43b8ce6f86" (UID: "3ef99af9-efaa-44ba-a64c-9b43b8ce6f86"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:00:41.622650 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.622584 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3ef99af9-efaa-44ba-a64c-9b43b8ce6f86" (UID: "3ef99af9-efaa-44ba-a64c-9b43b8ce6f86"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:00:41.622897 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.622864 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-service-ca" (OuterVolumeSpecName: "service-ca") pod "3ef99af9-efaa-44ba-a64c-9b43b8ce6f86" (UID: "3ef99af9-efaa-44ba-a64c-9b43b8ce6f86"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:00:41.622897 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.622887 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3ef99af9-efaa-44ba-a64c-9b43b8ce6f86" (UID: "3ef99af9-efaa-44ba-a64c-9b43b8ce6f86"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:00:41.624426 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.624400 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-kube-api-access-5nvcz" (OuterVolumeSpecName: "kube-api-access-5nvcz") pod "3ef99af9-efaa-44ba-a64c-9b43b8ce6f86" (UID: "3ef99af9-efaa-44ba-a64c-9b43b8ce6f86"). InnerVolumeSpecName "kube-api-access-5nvcz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:00:41.624426 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.624418 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3ef99af9-efaa-44ba-a64c-9b43b8ce6f86" (UID: "3ef99af9-efaa-44ba-a64c-9b43b8ce6f86"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:00:41.624570 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.624485 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3ef99af9-efaa-44ba-a64c-9b43b8ce6f86" (UID: "3ef99af9-efaa-44ba-a64c-9b43b8ce6f86"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:00:41.723740 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.723698 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nvcz\" (UniqueName: \"kubernetes.io/projected/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-kube-api-access-5nvcz\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:41.723740 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.723733 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-oauth-serving-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:41.723740 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.723743 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:41.723740 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.723752 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-console-oauth-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:41.724054 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.723761 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-trusted-ca-bundle\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:41.724054 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.723770 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-console-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:41.724054 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.723779 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86-console-serving-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:00:41.797098 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.797059 2574 generic.go:356] "Generic (PLEG): container finished" podID="3ef99af9-efaa-44ba-a64c-9b43b8ce6f86" containerID="9e6f2ce917e4ec8025e661ae871707184a9708dd7ad8fd25a247290afc9d5b80" exitCode=0 Jul 13 19:00:41.797251 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.797103 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7755b5d74f-gj8g9" Jul 13 19:00:41.797251 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.797113 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7755b5d74f-gj8g9" event={"ID":"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86","Type":"ContainerDied","Data":"9e6f2ce917e4ec8025e661ae871707184a9708dd7ad8fd25a247290afc9d5b80"} Jul 13 19:00:41.797251 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.797157 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7755b5d74f-gj8g9" event={"ID":"3ef99af9-efaa-44ba-a64c-9b43b8ce6f86","Type":"ContainerDied","Data":"813193f460d29da77cf46aacece3aeceddaff4a3da95f6e82e9dcc9bcc230761"} Jul 13 19:00:41.797251 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.797180 2574 scope.go:117] "RemoveContainer" containerID="9e6f2ce917e4ec8025e661ae871707184a9708dd7ad8fd25a247290afc9d5b80" Jul 13 19:00:41.806205 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.806187 2574 scope.go:117] "RemoveContainer" containerID="9e6f2ce917e4ec8025e661ae871707184a9708dd7ad8fd25a247290afc9d5b80" Jul 13 19:00:41.806495 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:00:41.806474 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e6f2ce917e4ec8025e661ae871707184a9708dd7ad8fd25a247290afc9d5b80\": container with ID starting with 9e6f2ce917e4ec8025e661ae871707184a9708dd7ad8fd25a247290afc9d5b80 not found: ID does not exist" containerID="9e6f2ce917e4ec8025e661ae871707184a9708dd7ad8fd25a247290afc9d5b80" Jul 13 19:00:41.806551 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.806504 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e6f2ce917e4ec8025e661ae871707184a9708dd7ad8fd25a247290afc9d5b80"} err="failed to get container status \"9e6f2ce917e4ec8025e661ae871707184a9708dd7ad8fd25a247290afc9d5b80\": rpc error: code = NotFound desc = could not find container \"9e6f2ce917e4ec8025e661ae871707184a9708dd7ad8fd25a247290afc9d5b80\": container with ID starting with 9e6f2ce917e4ec8025e661ae871707184a9708dd7ad8fd25a247290afc9d5b80 not found: ID does not exist" Jul 13 19:00:41.817654 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.817587 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7755b5d74f-gj8g9"] Jul 13 19:00:41.821458 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:41.821433 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7755b5d74f-gj8g9"] Jul 13 19:00:43.059704 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:43.059669 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ef99af9-efaa-44ba-a64c-9b43b8ce6f86" path="/var/lib/kubelet/pods/3ef99af9-efaa-44ba-a64c-9b43b8ce6f86/volumes" Jul 13 19:00:52.382667 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:52.382623 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:00:52.401834 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:52.401806 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:00:52.860139 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:00:52.860111 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:01.981410 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:01.981377 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:02.039927 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:02.039898 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:06.247297 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:06.247265 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 19:01:06.247679 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:06.247608 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="kube-rbac-proxy-metric" containerID="cri-o://ca9b4360b067d1259bae27ea2e1c0fc673f9a7c8d0e525a61e0429f88ce9ad16" gracePeriod=120 Jul 13 19:01:06.247679 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:06.247644 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="config-reloader" containerID="cri-o://e94a9a44b44aacd332cc2cfa3d3f662f54acee98d98bd90eaf3772757208fb92" gracePeriod=120 Jul 13 19:01:06.247817 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:06.247622 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="kube-rbac-proxy-web" containerID="cri-o://3946ff2516abf18b1eb04e95fa8dcad151319cbcb7a2bcb112e47608f54f6b8b" gracePeriod=120 Jul 13 19:01:06.247817 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:06.247591 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="alertmanager" containerID="cri-o://148a19b74445adc83dbdda78d3f94057f14771ede18849ec250d837aab58a066" gracePeriod=120 Jul 13 19:01:06.247817 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:06.247717 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="prom-label-proxy" containerID="cri-o://9bb595ed2d78e1db3fe1727645439052e79b5de625402e183a51982b5da7f664" gracePeriod=120 Jul 13 19:01:06.247817 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:06.247689 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="kube-rbac-proxy" containerID="cri-o://eadec454641cff51eda8d6e7652aa0bec1b58bf833a73d80461f658d60ca527f" gracePeriod=120 Jul 13 19:01:06.905490 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:06.905447 2574 generic.go:356] "Generic (PLEG): container finished" podID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerID="9bb595ed2d78e1db3fe1727645439052e79b5de625402e183a51982b5da7f664" exitCode=0 Jul 13 19:01:06.905490 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:06.905481 2574 generic.go:356] "Generic (PLEG): container finished" podID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerID="ca9b4360b067d1259bae27ea2e1c0fc673f9a7c8d0e525a61e0429f88ce9ad16" exitCode=0 Jul 13 19:01:06.905490 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:06.905488 2574 generic.go:356] "Generic (PLEG): container finished" podID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerID="eadec454641cff51eda8d6e7652aa0bec1b58bf833a73d80461f658d60ca527f" exitCode=0 Jul 13 19:01:06.905490 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:06.905495 2574 generic.go:356] "Generic (PLEG): container finished" podID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerID="e94a9a44b44aacd332cc2cfa3d3f662f54acee98d98bd90eaf3772757208fb92" exitCode=0 Jul 13 19:01:06.905834 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:06.905502 2574 generic.go:356] "Generic (PLEG): container finished" podID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerID="148a19b74445adc83dbdda78d3f94057f14771ede18849ec250d837aab58a066" exitCode=0 Jul 13 19:01:06.905834 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:06.905508 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a720e726-5e5b-44c4-83c7-06f5c5aa4554","Type":"ContainerDied","Data":"9bb595ed2d78e1db3fe1727645439052e79b5de625402e183a51982b5da7f664"} Jul 13 19:01:06.905834 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:06.905594 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a720e726-5e5b-44c4-83c7-06f5c5aa4554","Type":"ContainerDied","Data":"ca9b4360b067d1259bae27ea2e1c0fc673f9a7c8d0e525a61e0429f88ce9ad16"} Jul 13 19:01:06.905834 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:06.905637 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a720e726-5e5b-44c4-83c7-06f5c5aa4554","Type":"ContainerDied","Data":"eadec454641cff51eda8d6e7652aa0bec1b58bf833a73d80461f658d60ca527f"} Jul 13 19:01:06.905834 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:06.905673 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a720e726-5e5b-44c4-83c7-06f5c5aa4554","Type":"ContainerDied","Data":"e94a9a44b44aacd332cc2cfa3d3f662f54acee98d98bd90eaf3772757208fb92"} Jul 13 19:01:06.905834 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:06.905712 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a720e726-5e5b-44c4-83c7-06f5c5aa4554","Type":"ContainerDied","Data":"148a19b74445adc83dbdda78d3f94057f14771ede18849ec250d837aab58a066"} Jul 13 19:01:07.596912 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.596885 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:07.760936 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.760848 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a720e726-5e5b-44c4-83c7-06f5c5aa4554-alertmanager-main-db\") pod \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " Jul 13 19:01:07.760936 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.760896 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-web-config\") pod \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " Jul 13 19:01:07.760936 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.760926 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-config-volume\") pod \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " Jul 13 19:01:07.761225 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.760947 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-cluster-tls-config\") pod \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " Jul 13 19:01:07.761225 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.760964 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4nz4\" (UniqueName: \"kubernetes.io/projected/a720e726-5e5b-44c4-83c7-06f5c5aa4554-kube-api-access-d4nz4\") pod \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " Jul 13 19:01:07.761225 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.760986 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-kube-rbac-proxy-metric\") pod \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " Jul 13 19:01:07.761225 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.761106 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a720e726-5e5b-44c4-83c7-06f5c5aa4554-tls-assets\") pod \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " Jul 13 19:01:07.761225 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.761142 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a720e726-5e5b-44c4-83c7-06f5c5aa4554-config-out\") pod \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " Jul 13 19:01:07.761225 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.761170 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-kube-rbac-proxy\") pod \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " Jul 13 19:01:07.761225 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.761202 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a720e726-5e5b-44c4-83c7-06f5c5aa4554-alertmanager-trusted-ca-bundle\") pod \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " Jul 13 19:01:07.761225 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.761215 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a720e726-5e5b-44c4-83c7-06f5c5aa4554-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "a720e726-5e5b-44c4-83c7-06f5c5aa4554" (UID: "a720e726-5e5b-44c4-83c7-06f5c5aa4554"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:01:07.761631 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.761236 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-main-tls\") pod \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " Jul 13 19:01:07.761631 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.761273 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a720e726-5e5b-44c4-83c7-06f5c5aa4554-metrics-client-ca\") pod \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " Jul 13 19:01:07.761631 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.761299 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-kube-rbac-proxy-web\") pod \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\" (UID: \"a720e726-5e5b-44c4-83c7-06f5c5aa4554\") " Jul 13 19:01:07.761631 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.761602 2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a720e726-5e5b-44c4-83c7-06f5c5aa4554-alertmanager-main-db\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:07.764339 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.764282 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a720e726-5e5b-44c4-83c7-06f5c5aa4554-config-out" (OuterVolumeSpecName: "config-out") pod "a720e726-5e5b-44c4-83c7-06f5c5aa4554" (UID: "a720e726-5e5b-44c4-83c7-06f5c5aa4554"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:01:07.764681 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.764402 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a720e726-5e5b-44c4-83c7-06f5c5aa4554-kube-api-access-d4nz4" (OuterVolumeSpecName: "kube-api-access-d4nz4") pod "a720e726-5e5b-44c4-83c7-06f5c5aa4554" (UID: "a720e726-5e5b-44c4-83c7-06f5c5aa4554"). InnerVolumeSpecName "kube-api-access-d4nz4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:01:07.764681 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.764541 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a720e726-5e5b-44c4-83c7-06f5c5aa4554-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a720e726-5e5b-44c4-83c7-06f5c5aa4554" (UID: "a720e726-5e5b-44c4-83c7-06f5c5aa4554"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:01:07.764681 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.764295 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "a720e726-5e5b-44c4-83c7-06f5c5aa4554" (UID: "a720e726-5e5b-44c4-83c7-06f5c5aa4554"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:01:07.764895 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.764682 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "a720e726-5e5b-44c4-83c7-06f5c5aa4554" (UID: "a720e726-5e5b-44c4-83c7-06f5c5aa4554"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:01:07.764895 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.764798 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a720e726-5e5b-44c4-83c7-06f5c5aa4554-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "a720e726-5e5b-44c4-83c7-06f5c5aa4554" (UID: "a720e726-5e5b-44c4-83c7-06f5c5aa4554"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:01:07.764895 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.764863 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-config-volume" (OuterVolumeSpecName: "config-volume") pod "a720e726-5e5b-44c4-83c7-06f5c5aa4554" (UID: "a720e726-5e5b-44c4-83c7-06f5c5aa4554"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:01:07.764895 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.764874 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a720e726-5e5b-44c4-83c7-06f5c5aa4554-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "a720e726-5e5b-44c4-83c7-06f5c5aa4554" (UID: "a720e726-5e5b-44c4-83c7-06f5c5aa4554"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:01:07.766500 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.766480 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "a720e726-5e5b-44c4-83c7-06f5c5aa4554" (UID: "a720e726-5e5b-44c4-83c7-06f5c5aa4554"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:01:07.766601 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.766583 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "a720e726-5e5b-44c4-83c7-06f5c5aa4554" (UID: "a720e726-5e5b-44c4-83c7-06f5c5aa4554"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:01:07.769663 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.769636 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "a720e726-5e5b-44c4-83c7-06f5c5aa4554" (UID: "a720e726-5e5b-44c4-83c7-06f5c5aa4554"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:01:07.776379 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.776356 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-web-config" (OuterVolumeSpecName: "web-config") pod "a720e726-5e5b-44c4-83c7-06f5c5aa4554" (UID: "a720e726-5e5b-44c4-83c7-06f5c5aa4554"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:01:07.862939 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.862912 2574 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-cluster-tls-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:07.862939 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.862935 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4nz4\" (UniqueName: \"kubernetes.io/projected/a720e726-5e5b-44c4-83c7-06f5c5aa4554-kube-api-access-d4nz4\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:07.862939 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.862945 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:07.863164 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.862954 2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a720e726-5e5b-44c4-83c7-06f5c5aa4554-tls-assets\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:07.863164 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.862965 2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a720e726-5e5b-44c4-83c7-06f5c5aa4554-config-out\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:07.863164 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.862973 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:07.863164 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.862982 2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a720e726-5e5b-44c4-83c7-06f5c5aa4554-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:07.863164 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.862991 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-main-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:07.863164 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.862999 2574 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a720e726-5e5b-44c4-83c7-06f5c5aa4554-metrics-client-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:07.863164 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.863007 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:07.863164 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.863016 2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-web-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:07.863164 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.863024 2574 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a720e726-5e5b-44c4-83c7-06f5c5aa4554-config-volume\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:07.912543 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.912508 2574 generic.go:356] "Generic (PLEG): container finished" podID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerID="3946ff2516abf18b1eb04e95fa8dcad151319cbcb7a2bcb112e47608f54f6b8b" exitCode=0 Jul 13 19:01:07.912719 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.912553 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a720e726-5e5b-44c4-83c7-06f5c5aa4554","Type":"ContainerDied","Data":"3946ff2516abf18b1eb04e95fa8dcad151319cbcb7a2bcb112e47608f54f6b8b"} Jul 13 19:01:07.912719 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.912605 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:07.912719 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.912627 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a720e726-5e5b-44c4-83c7-06f5c5aa4554","Type":"ContainerDied","Data":"59cf82e73228a021418f21ee33a5781782a32f489351c8fb3f078796c40a6aab"} Jul 13 19:01:07.912719 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.912674 2574 scope.go:117] "RemoveContainer" containerID="9bb595ed2d78e1db3fe1727645439052e79b5de625402e183a51982b5da7f664" Jul 13 19:01:07.921126 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.921104 2574 scope.go:117] "RemoveContainer" containerID="ca9b4360b067d1259bae27ea2e1c0fc673f9a7c8d0e525a61e0429f88ce9ad16" Jul 13 19:01:07.928348 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.928327 2574 scope.go:117] "RemoveContainer" containerID="eadec454641cff51eda8d6e7652aa0bec1b58bf833a73d80461f658d60ca527f" Jul 13 19:01:07.935859 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.935835 2574 scope.go:117] "RemoveContainer" containerID="3946ff2516abf18b1eb04e95fa8dcad151319cbcb7a2bcb112e47608f54f6b8b" Jul 13 19:01:07.937481 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.937459 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 19:01:07.941522 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.941501 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 19:01:07.943741 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.943727 2574 scope.go:117] "RemoveContainer" containerID="e94a9a44b44aacd332cc2cfa3d3f662f54acee98d98bd90eaf3772757208fb92" Jul 13 19:01:07.950881 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.950862 2574 scope.go:117] "RemoveContainer" containerID="148a19b74445adc83dbdda78d3f94057f14771ede18849ec250d837aab58a066" Jul 13 19:01:07.957900 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.957875 2574 scope.go:117] "RemoveContainer" containerID="b1957ff012aa4425ce169a811355aaf37d33128ba982e8b6b4f833e8c6697fd8" Jul 13 19:01:07.965624 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.965606 2574 scope.go:117] "RemoveContainer" containerID="9bb595ed2d78e1db3fe1727645439052e79b5de625402e183a51982b5da7f664" Jul 13 19:01:07.965917 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:01:07.965897 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bb595ed2d78e1db3fe1727645439052e79b5de625402e183a51982b5da7f664\": container with ID starting with 9bb595ed2d78e1db3fe1727645439052e79b5de625402e183a51982b5da7f664 not found: ID does not exist" containerID="9bb595ed2d78e1db3fe1727645439052e79b5de625402e183a51982b5da7f664" Jul 13 19:01:07.965979 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.965927 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bb595ed2d78e1db3fe1727645439052e79b5de625402e183a51982b5da7f664"} err="failed to get container status \"9bb595ed2d78e1db3fe1727645439052e79b5de625402e183a51982b5da7f664\": rpc error: code = NotFound desc = could not find container \"9bb595ed2d78e1db3fe1727645439052e79b5de625402e183a51982b5da7f664\": container with ID starting with 9bb595ed2d78e1db3fe1727645439052e79b5de625402e183a51982b5da7f664 not found: ID does not exist" Jul 13 19:01:07.965979 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.965948 2574 scope.go:117] "RemoveContainer" containerID="ca9b4360b067d1259bae27ea2e1c0fc673f9a7c8d0e525a61e0429f88ce9ad16" Jul 13 19:01:07.966189 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:01:07.966171 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca9b4360b067d1259bae27ea2e1c0fc673f9a7c8d0e525a61e0429f88ce9ad16\": container with ID starting with ca9b4360b067d1259bae27ea2e1c0fc673f9a7c8d0e525a61e0429f88ce9ad16 not found: ID does not exist" containerID="ca9b4360b067d1259bae27ea2e1c0fc673f9a7c8d0e525a61e0429f88ce9ad16" Jul 13 19:01:07.966251 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.966199 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca9b4360b067d1259bae27ea2e1c0fc673f9a7c8d0e525a61e0429f88ce9ad16"} err="failed to get container status \"ca9b4360b067d1259bae27ea2e1c0fc673f9a7c8d0e525a61e0429f88ce9ad16\": rpc error: code = NotFound desc = could not find container \"ca9b4360b067d1259bae27ea2e1c0fc673f9a7c8d0e525a61e0429f88ce9ad16\": container with ID starting with ca9b4360b067d1259bae27ea2e1c0fc673f9a7c8d0e525a61e0429f88ce9ad16 not found: ID does not exist" Jul 13 19:01:07.966251 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.966223 2574 scope.go:117] "RemoveContainer" containerID="eadec454641cff51eda8d6e7652aa0bec1b58bf833a73d80461f658d60ca527f" Jul 13 19:01:07.966468 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:01:07.966453 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eadec454641cff51eda8d6e7652aa0bec1b58bf833a73d80461f658d60ca527f\": container with ID starting with eadec454641cff51eda8d6e7652aa0bec1b58bf833a73d80461f658d60ca527f not found: ID does not exist" containerID="eadec454641cff51eda8d6e7652aa0bec1b58bf833a73d80461f658d60ca527f" Jul 13 19:01:07.966506 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.966471 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eadec454641cff51eda8d6e7652aa0bec1b58bf833a73d80461f658d60ca527f"} err="failed to get container status \"eadec454641cff51eda8d6e7652aa0bec1b58bf833a73d80461f658d60ca527f\": rpc error: code = NotFound desc = could not find container \"eadec454641cff51eda8d6e7652aa0bec1b58bf833a73d80461f658d60ca527f\": container with ID starting with eadec454641cff51eda8d6e7652aa0bec1b58bf833a73d80461f658d60ca527f not found: ID does not exist" Jul 13 19:01:07.966506 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.966484 2574 scope.go:117] "RemoveContainer" containerID="3946ff2516abf18b1eb04e95fa8dcad151319cbcb7a2bcb112e47608f54f6b8b" Jul 13 19:01:07.966705 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:01:07.966688 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3946ff2516abf18b1eb04e95fa8dcad151319cbcb7a2bcb112e47608f54f6b8b\": container with ID starting with 3946ff2516abf18b1eb04e95fa8dcad151319cbcb7a2bcb112e47608f54f6b8b not found: ID does not exist" containerID="3946ff2516abf18b1eb04e95fa8dcad151319cbcb7a2bcb112e47608f54f6b8b" Jul 13 19:01:07.966773 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.966711 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3946ff2516abf18b1eb04e95fa8dcad151319cbcb7a2bcb112e47608f54f6b8b"} err="failed to get container status \"3946ff2516abf18b1eb04e95fa8dcad151319cbcb7a2bcb112e47608f54f6b8b\": rpc error: code = NotFound desc = could not find container \"3946ff2516abf18b1eb04e95fa8dcad151319cbcb7a2bcb112e47608f54f6b8b\": container with ID starting with 3946ff2516abf18b1eb04e95fa8dcad151319cbcb7a2bcb112e47608f54f6b8b not found: ID does not exist" Jul 13 19:01:07.966773 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.966732 2574 scope.go:117] "RemoveContainer" containerID="e94a9a44b44aacd332cc2cfa3d3f662f54acee98d98bd90eaf3772757208fb92" Jul 13 19:01:07.966983 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:01:07.966966 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e94a9a44b44aacd332cc2cfa3d3f662f54acee98d98bd90eaf3772757208fb92\": container with ID starting with e94a9a44b44aacd332cc2cfa3d3f662f54acee98d98bd90eaf3772757208fb92 not found: ID does not exist" containerID="e94a9a44b44aacd332cc2cfa3d3f662f54acee98d98bd90eaf3772757208fb92" Jul 13 19:01:07.967021 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.966988 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e94a9a44b44aacd332cc2cfa3d3f662f54acee98d98bd90eaf3772757208fb92"} err="failed to get container status \"e94a9a44b44aacd332cc2cfa3d3f662f54acee98d98bd90eaf3772757208fb92\": rpc error: code = NotFound desc = could not find container \"e94a9a44b44aacd332cc2cfa3d3f662f54acee98d98bd90eaf3772757208fb92\": container with ID starting with e94a9a44b44aacd332cc2cfa3d3f662f54acee98d98bd90eaf3772757208fb92 not found: ID does not exist" Jul 13 19:01:07.967021 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.967003 2574 scope.go:117] "RemoveContainer" containerID="148a19b74445adc83dbdda78d3f94057f14771ede18849ec250d837aab58a066" Jul 13 19:01:07.967262 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:01:07.967245 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"148a19b74445adc83dbdda78d3f94057f14771ede18849ec250d837aab58a066\": container with ID starting with 148a19b74445adc83dbdda78d3f94057f14771ede18849ec250d837aab58a066 not found: ID does not exist" containerID="148a19b74445adc83dbdda78d3f94057f14771ede18849ec250d837aab58a066" Jul 13 19:01:07.967314 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.967265 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"148a19b74445adc83dbdda78d3f94057f14771ede18849ec250d837aab58a066"} err="failed to get container status \"148a19b74445adc83dbdda78d3f94057f14771ede18849ec250d837aab58a066\": rpc error: code = NotFound desc = could not find container \"148a19b74445adc83dbdda78d3f94057f14771ede18849ec250d837aab58a066\": container with ID starting with 148a19b74445adc83dbdda78d3f94057f14771ede18849ec250d837aab58a066 not found: ID does not exist" Jul 13 19:01:07.967314 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.967278 2574 scope.go:117] "RemoveContainer" containerID="b1957ff012aa4425ce169a811355aaf37d33128ba982e8b6b4f833e8c6697fd8" Jul 13 19:01:07.967510 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:01:07.967445 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1957ff012aa4425ce169a811355aaf37d33128ba982e8b6b4f833e8c6697fd8\": container with ID starting with b1957ff012aa4425ce169a811355aaf37d33128ba982e8b6b4f833e8c6697fd8 not found: ID does not exist" containerID="b1957ff012aa4425ce169a811355aaf37d33128ba982e8b6b4f833e8c6697fd8" Jul 13 19:01:07.967510 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.967461 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1957ff012aa4425ce169a811355aaf37d33128ba982e8b6b4f833e8c6697fd8"} err="failed to get container status \"b1957ff012aa4425ce169a811355aaf37d33128ba982e8b6b4f833e8c6697fd8\": rpc error: code = NotFound desc = could not find container \"b1957ff012aa4425ce169a811355aaf37d33128ba982e8b6b4f833e8c6697fd8\": container with ID starting with b1957ff012aa4425ce169a811355aaf37d33128ba982e8b6b4f833e8c6697fd8 not found: ID does not exist" Jul 13 19:01:07.976445 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976419 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 19:01:07.976727 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976716 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="kube-rbac-proxy" Jul 13 19:01:07.976838 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976729 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="kube-rbac-proxy" Jul 13 19:01:07.976838 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976738 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99241bde-9e5c-428e-9714-bf8b500410c2" containerName="console" Jul 13 19:01:07.976838 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976743 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="99241bde-9e5c-428e-9714-bf8b500410c2" containerName="console" Jul 13 19:01:07.976838 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976750 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="config-reloader" Jul 13 19:01:07.976838 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976756 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="config-reloader" Jul 13 19:01:07.976838 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976765 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="kube-rbac-proxy-web" Jul 13 19:01:07.976838 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976770 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="kube-rbac-proxy-web" Jul 13 19:01:07.976838 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976778 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="kube-rbac-proxy-metric" Jul 13 19:01:07.976838 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976800 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="kube-rbac-proxy-metric" Jul 13 19:01:07.976838 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976808 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c8bb84e-6492-4673-a8e1-d20f38615d40" containerName="console" Jul 13 19:01:07.976838 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976813 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c8bb84e-6492-4673-a8e1-d20f38615d40" containerName="console" Jul 13 19:01:07.976838 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976821 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="init-config-reloader" Jul 13 19:01:07.976838 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976826 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="init-config-reloader" Jul 13 19:01:07.976838 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976833 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="prom-label-proxy" Jul 13 19:01:07.976838 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976838 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="prom-label-proxy" Jul 13 19:01:07.976838 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976843 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ef99af9-efaa-44ba-a64c-9b43b8ce6f86" containerName="console" Jul 13 19:01:07.977282 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976849 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ef99af9-efaa-44ba-a64c-9b43b8ce6f86" containerName="console" Jul 13 19:01:07.977282 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976856 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="alertmanager" Jul 13 19:01:07.977282 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976863 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="alertmanager" Jul 13 19:01:07.977282 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976921 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="kube-rbac-proxy" Jul 13 19:01:07.977282 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976930 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c8bb84e-6492-4673-a8e1-d20f38615d40" containerName="console" Jul 13 19:01:07.977282 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976937 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ef99af9-efaa-44ba-a64c-9b43b8ce6f86" containerName="console" Jul 13 19:01:07.977282 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976942 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="alertmanager" Jul 13 19:01:07.977282 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976947 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="kube-rbac-proxy-web" Jul 13 19:01:07.977282 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976953 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="config-reloader" Jul 13 19:01:07.977282 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976959 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="prom-label-proxy" Jul 13 19:01:07.977282 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976963 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" containerName="kube-rbac-proxy-metric" Jul 13 19:01:07.977282 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.976970 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="99241bde-9e5c-428e-9714-bf8b500410c2" containerName="console" Jul 13 19:01:07.981003 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.980987 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:07.983230 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.983208 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 13 19:01:07.983441 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.983231 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4jwgl\"" Jul 13 19:01:07.983441 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.983240 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 13 19:01:07.983441 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.983247 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 13 19:01:07.983441 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.983208 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 13 19:01:07.983644 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.983539 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 13 19:01:07.983697 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.983653 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 13 19:01:07.983745 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.983693 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 13 19:01:07.983960 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.983945 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 13 19:01:07.988582 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.988477 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 13 19:01:07.993465 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:07.993442 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 19:01:08.064632 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.064600 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-web-config\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.064632 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.064633 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.064892 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.064653 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.064892 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.064672 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.064892 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.064747 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n8fq\" (UniqueName: \"kubernetes.io/projected/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-kube-api-access-4n8fq\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.064892 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.064793 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-config-volume\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.064892 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.064842 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.065171 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.064897 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.065171 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.064938 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.065171 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.064967 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-config-out\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.065171 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.064994 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.065171 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.065025 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.065171 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.065043 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.165979 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.165937 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-config-volume\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.166148 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.165986 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.166148 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.166019 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.166148 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.166048 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.166148 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.166072 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-config-out\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.166148 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.166097 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.166404 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.166231 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.166404 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.166273 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.166404 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.166351 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-web-config\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.166404 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.166376 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.166611 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.166404 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.166611 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.166428 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.166611 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.166475 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n8fq\" (UniqueName: \"kubernetes.io/projected/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-kube-api-access-4n8fq\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.167245 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.167192 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.167411 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.167274 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.167506 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.167490 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.169488 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.169452 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.169593 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.169526 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.169593 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.169528 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.169593 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.169563 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.169812 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.169764 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-config-out\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.169964 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.169940 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.170021 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.169946 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-config-volume\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.170064 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.170040 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-web-config\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.170729 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.170708 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.174745 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.174726 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n8fq\" (UniqueName: \"kubernetes.io/projected/7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f-kube-api-access-4n8fq\") pod \"alertmanager-main-0\" (UID: \"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.291239 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.291204 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:01:08.427152 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.427101 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 19:01:08.429125 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:01:08.429091 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b057d6f_c5a5_4d8b_b9fa_f1b0056a424f.slice/crio-1a017bb531065d034ae10c13d360e44769b6ebc415fe8a4acf27de731c9699a8 WatchSource:0}: Error finding container 1a017bb531065d034ae10c13d360e44769b6ebc415fe8a4acf27de731c9699a8: Status 404 returned error can't find the container with id 1a017bb531065d034ae10c13d360e44769b6ebc415fe8a4acf27de731c9699a8 Jul 13 19:01:08.918055 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.918024 2574 generic.go:356] "Generic (PLEG): container finished" podID="7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f" containerID="9b3180b5e98f264e1a6471c4eab1c780816c3af8eacce5a653e4d78bad0afa66" exitCode=0 Jul 13 19:01:08.918467 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.918121 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f","Type":"ContainerDied","Data":"9b3180b5e98f264e1a6471c4eab1c780816c3af8eacce5a653e4d78bad0afa66"} Jul 13 19:01:08.918467 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:08.918214 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f","Type":"ContainerStarted","Data":"1a017bb531065d034ae10c13d360e44769b6ebc415fe8a4acf27de731c9699a8"} Jul 13 19:01:09.060699 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:09.060671 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a720e726-5e5b-44c4-83c7-06f5c5aa4554" path="/var/lib/kubelet/pods/a720e726-5e5b-44c4-83c7-06f5c5aa4554/volumes" Jul 13 19:01:09.927379 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:09.927342 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f","Type":"ContainerStarted","Data":"af9ed97fdff63cc144a3751b09a06af64362003299b48331abf3fe917745752f"} Jul 13 19:01:09.927845 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:09.927406 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f","Type":"ContainerStarted","Data":"681307ca4c4aad776af689bb48e9d92d6d28d17aa6710ae5efce3836a4128fac"} Jul 13 19:01:09.927845 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:09.927430 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f","Type":"ContainerStarted","Data":"27c073e98ee0b8cef336a11b47cb8d947fd48589553b2041b3c9eda4221f3f1d"} Jul 13 19:01:09.927845 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:09.927457 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f","Type":"ContainerStarted","Data":"726e287c2ad2f5938604874f2346391e5d2cdbe4d8051231746cb198cfba90ff"} Jul 13 19:01:09.927845 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:09.927481 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f","Type":"ContainerStarted","Data":"9afd6cd1ebaf0d175c42b20e2c144cdf8b4f1ae3864a73d2eab954fd5089ca8a"} Jul 13 19:01:09.927845 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:09.927504 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f","Type":"ContainerStarted","Data":"cb07f6146c29dadc13daa2ba3347790b8c6fff470869fa98545b63fa0d0f6a6f"} Jul 13 19:01:09.955798 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:09.955726 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.939030879 podStartE2EDuration="2.955710033s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.679154ms" isStatefulPod=false podCreationTimestamp="2026-07-13 19:01:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:01:09.953373172 +0000 UTC m=+155.429827347" watchObservedRunningTime="2026-07-13 19:01:09.955710033 +0000 UTC m=+155.432164207" Jul 13 19:01:10.482475 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.482445 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 19:01:10.483088 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.482863 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="kube-rbac-proxy-web" containerID="cri-o://a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179" gracePeriod=600 Jul 13 19:01:10.483088 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.482904 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="config-reloader" containerID="cri-o://0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd" gracePeriod=600 Jul 13 19:01:10.483088 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.482846 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="kube-rbac-proxy-thanos" containerID="cri-o://47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4" gracePeriod=600 Jul 13 19:01:10.483088 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.482871 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="kube-rbac-proxy" containerID="cri-o://4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9" gracePeriod=600 Jul 13 19:01:10.483088 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.482869 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="prometheus" containerID="cri-o://c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1" gracePeriod=600 Jul 13 19:01:10.483441 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.483114 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="thanos-sidecar" containerID="cri-o://03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec" gracePeriod=600 Jul 13 19:01:10.734507 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.734437 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:10.889860 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.889828 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890045 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.889880 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-rulefiles-1\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890045 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.889904 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khxqd\" (UniqueName: \"kubernetes.io/projected/e1e1de81-a666-4de5-976f-5ac4a6ec9235-kube-api-access-khxqd\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890045 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.889923 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-kube-rbac-proxy\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890045 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.889953 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-rulefiles-2\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890045 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.889975 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e1e1de81-a666-4de5-976f-5ac4a6ec9235-tls-assets\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890045 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.890006 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890045 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.890034 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-web-config\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890435 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.890084 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-configmap-kubelet-serving-ca-bundle\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890435 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.890127 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-config\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890435 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.890166 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-db\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890435 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.890193 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-rulefiles-0\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890435 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.890217 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-grpc-tls\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890435 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.890243 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-prometheus-k8s-tls\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890435 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.890266 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-configmap-serving-certs-ca-bundle\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890435 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.890298 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-configmap-metrics-client-ca\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890435 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.890292 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:01:10.890435 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.890341 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-trusted-ca-bundle\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890435 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.890367 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-metrics-client-certs\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890435 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.890400 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e1e1de81-a666-4de5-976f-5ac4a6ec9235-config-out\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.890435 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.890429 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-thanos-prometheus-http-client-file\") pod \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\" (UID: \"e1e1de81-a666-4de5-976f-5ac4a6ec9235\") " Jul 13 19:01:10.891092 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.890633 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:01:10.891092 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.890749 2574 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.891092 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.890768 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.891381 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.891356 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:01:10.892633 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.892606 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:01:10.892905 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.892872 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:01:10.892905 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.892951 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:01:10.893224 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.893186 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:01:10.893224 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.893189 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1e1de81-a666-4de5-976f-5ac4a6ec9235-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:01:10.893582 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.893463 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:01:10.893582 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.893508 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1e1de81-a666-4de5-976f-5ac4a6ec9235-kube-api-access-khxqd" (OuterVolumeSpecName: "kube-api-access-khxqd") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "kube-api-access-khxqd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:01:10.893582 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.893532 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:01:10.894039 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.893971 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-config" (OuterVolumeSpecName: "config") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:01:10.894241 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.894099 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:01:10.894315 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.894285 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:01:10.895611 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.895576 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:01:10.895766 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.895724 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:01:10.897247 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.897218 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:01:10.897247 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.897235 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1e1de81-a666-4de5-976f-5ac4a6ec9235-config-out" (OuterVolumeSpecName: "config-out") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:01:10.897444 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.897430 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:01:10.906489 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.906464 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-web-config" (OuterVolumeSpecName: "web-config") pod "e1e1de81-a666-4de5-976f-5ac4a6ec9235" (UID: "e1e1de81-a666-4de5-976f-5ac4a6ec9235"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:01:10.935013 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.934982 2574 generic.go:356] "Generic (PLEG): container finished" podID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerID="47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4" exitCode=0 Jul 13 19:01:10.935013 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.935007 2574 generic.go:356] "Generic (PLEG): container finished" podID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerID="4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9" exitCode=0 Jul 13 19:01:10.935013 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.935013 2574 generic.go:356] "Generic (PLEG): container finished" podID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerID="a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179" exitCode=0 Jul 13 19:01:10.935013 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.935019 2574 generic.go:356] "Generic (PLEG): container finished" podID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerID="03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec" exitCode=0 Jul 13 19:01:10.935559 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.935025 2574 generic.go:356] "Generic (PLEG): container finished" podID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerID="0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd" exitCode=0 Jul 13 19:01:10.935559 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.935031 2574 generic.go:356] "Generic (PLEG): container finished" podID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerID="c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1" exitCode=0 Jul 13 19:01:10.935559 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.935057 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:10.935559 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.935061 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1e1de81-a666-4de5-976f-5ac4a6ec9235","Type":"ContainerDied","Data":"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4"} Jul 13 19:01:10.935559 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.935156 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1e1de81-a666-4de5-976f-5ac4a6ec9235","Type":"ContainerDied","Data":"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9"} Jul 13 19:01:10.935559 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.935202 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1e1de81-a666-4de5-976f-5ac4a6ec9235","Type":"ContainerDied","Data":"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179"} Jul 13 19:01:10.935559 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.935216 2574 scope.go:117] "RemoveContainer" containerID="47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4" Jul 13 19:01:10.935559 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.935244 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1e1de81-a666-4de5-976f-5ac4a6ec9235","Type":"ContainerDied","Data":"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec"} Jul 13 19:01:10.935559 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.935288 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1e1de81-a666-4de5-976f-5ac4a6ec9235","Type":"ContainerDied","Data":"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd"} Jul 13 19:01:10.935559 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.935334 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1e1de81-a666-4de5-976f-5ac4a6ec9235","Type":"ContainerDied","Data":"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1"} Jul 13 19:01:10.935559 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.935372 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1e1de81-a666-4de5-976f-5ac4a6ec9235","Type":"ContainerDied","Data":"18c2f4c1d365bcff205807a749d7fe0d8db8f4d95e290ea782a6110a47e0f50f"} Jul 13 19:01:10.943708 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.943689 2574 scope.go:117] "RemoveContainer" containerID="4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9" Jul 13 19:01:10.950710 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.950694 2574 scope.go:117] "RemoveContainer" containerID="a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179" Jul 13 19:01:10.957445 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.957427 2574 scope.go:117] "RemoveContainer" containerID="03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec" Jul 13 19:01:10.960769 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.960748 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 19:01:10.965612 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.965590 2574 scope.go:117] "RemoveContainer" containerID="0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd" Jul 13 19:01:10.966409 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.966387 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 19:01:10.972249 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.972230 2574 scope.go:117] "RemoveContainer" containerID="c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1" Jul 13 19:01:10.979373 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.979359 2574 scope.go:117] "RemoveContainer" containerID="e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26" Jul 13 19:01:10.986185 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.986166 2574 scope.go:117] "RemoveContainer" containerID="47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4" Jul 13 19:01:10.986448 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:01:10.986414 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4\": container with ID starting with 47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4 not found: ID does not exist" containerID="47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4" Jul 13 19:01:10.986516 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.986460 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4"} err="failed to get container status \"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4\": rpc error: code = NotFound desc = could not find container \"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4\": container with ID starting with 47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4 not found: ID does not exist" Jul 13 19:01:10.986516 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.986488 2574 scope.go:117] "RemoveContainer" containerID="4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9" Jul 13 19:01:10.986716 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:01:10.986698 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9\": container with ID starting with 4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9 not found: ID does not exist" containerID="4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9" Jul 13 19:01:10.986754 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.986722 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9"} err="failed to get container status \"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9\": rpc error: code = NotFound desc = could not find container \"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9\": container with ID starting with 4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9 not found: ID does not exist" Jul 13 19:01:10.986754 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.986743 2574 scope.go:117] "RemoveContainer" containerID="a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179" Jul 13 19:01:10.986967 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:01:10.986950 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179\": container with ID starting with a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179 not found: ID does not exist" containerID="a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179" Jul 13 19:01:10.987030 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.986977 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179"} err="failed to get container status \"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179\": rpc error: code = NotFound desc = could not find container \"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179\": container with ID starting with a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179 not found: ID does not exist" Jul 13 19:01:10.987030 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.986998 2574 scope.go:117] "RemoveContainer" containerID="03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec" Jul 13 19:01:10.987281 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:01:10.987263 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec\": container with ID starting with 03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec not found: ID does not exist" containerID="03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec" Jul 13 19:01:10.987317 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.987288 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec"} err="failed to get container status \"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec\": rpc error: code = NotFound desc = could not find container \"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec\": container with ID starting with 03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec not found: ID does not exist" Jul 13 19:01:10.987317 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.987305 2574 scope.go:117] "RemoveContainer" containerID="0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd" Jul 13 19:01:10.987523 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:01:10.987506 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd\": container with ID starting with 0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd not found: ID does not exist" containerID="0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd" Jul 13 19:01:10.987610 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.987529 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd"} err="failed to get container status \"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd\": rpc error: code = NotFound desc = could not find container \"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd\": container with ID starting with 0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd not found: ID does not exist" Jul 13 19:01:10.987610 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.987548 2574 scope.go:117] "RemoveContainer" containerID="c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1" Jul 13 19:01:10.987777 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:01:10.987759 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1\": container with ID starting with c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1 not found: ID does not exist" containerID="c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1" Jul 13 19:01:10.987777 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.987794 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1"} err="failed to get container status \"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1\": rpc error: code = NotFound desc = could not find container \"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1\": container with ID starting with c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1 not found: ID does not exist" Jul 13 19:01:10.987777 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.987812 2574 scope.go:117] "RemoveContainer" containerID="e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26" Jul 13 19:01:10.988072 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:01:10.988055 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26\": container with ID starting with e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26 not found: ID does not exist" containerID="e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26" Jul 13 19:01:10.988138 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.988079 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26"} err="failed to get container status \"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26\": rpc error: code = NotFound desc = could not find container \"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26\": container with ID starting with e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26 not found: ID does not exist" Jul 13 19:01:10.988138 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.988098 2574 scope.go:117] "RemoveContainer" containerID="47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4" Jul 13 19:01:10.988316 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.988301 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4"} err="failed to get container status \"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4\": rpc error: code = NotFound desc = could not find container \"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4\": container with ID starting with 47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4 not found: ID does not exist" Jul 13 19:01:10.988364 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.988317 2574 scope.go:117] "RemoveContainer" containerID="4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9" Jul 13 19:01:10.988518 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.988504 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9"} err="failed to get container status \"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9\": rpc error: code = NotFound desc = could not find container \"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9\": container with ID starting with 4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9 not found: ID does not exist" Jul 13 19:01:10.988564 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.988517 2574 scope.go:117] "RemoveContainer" containerID="a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179" Jul 13 19:01:10.988713 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.988692 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179"} err="failed to get container status \"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179\": rpc error: code = NotFound desc = could not find container \"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179\": container with ID starting with a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179 not found: ID does not exist" Jul 13 19:01:10.988776 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.988714 2574 scope.go:117] "RemoveContainer" containerID="03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec" Jul 13 19:01:10.989306 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.989259 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec"} err="failed to get container status \"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec\": rpc error: code = NotFound desc = could not find container \"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec\": container with ID starting with 03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec not found: ID does not exist" Jul 13 19:01:10.989461 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.989308 2574 scope.go:117] "RemoveContainer" containerID="0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd" Jul 13 19:01:10.989685 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.989661 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd"} err="failed to get container status \"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd\": rpc error: code = NotFound desc = could not find container \"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd\": container with ID starting with 0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd not found: ID does not exist" Jul 13 19:01:10.989685 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.989684 2574 scope.go:117] "RemoveContainer" containerID="c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1" Jul 13 19:01:10.990007 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.989970 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1"} err="failed to get container status \"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1\": rpc error: code = NotFound desc = could not find container \"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1\": container with ID starting with c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1 not found: ID does not exist" Jul 13 19:01:10.990007 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.989995 2574 scope.go:117] "RemoveContainer" containerID="e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26" Jul 13 19:01:10.990519 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.990500 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26"} err="failed to get container status \"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26\": rpc error: code = NotFound desc = could not find container \"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26\": container with ID starting with e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26 not found: ID does not exist" Jul 13 19:01:10.990564 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.990520 2574 scope.go:117] "RemoveContainer" containerID="47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4" Jul 13 19:01:10.990811 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.990758 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4"} err="failed to get container status \"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4\": rpc error: code = NotFound desc = could not find container \"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4\": container with ID starting with 47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4 not found: ID does not exist" Jul 13 19:01:10.990901 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.990825 2574 scope.go:117] "RemoveContainer" containerID="4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9" Jul 13 19:01:10.991080 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991057 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9"} err="failed to get container status \"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9\": rpc error: code = NotFound desc = could not find container \"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9\": container with ID starting with 4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9 not found: ID does not exist" Jul 13 19:01:10.991080 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991075 2574 scope.go:117] "RemoveContainer" containerID="a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179" Jul 13 19:01:10.991267 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991250 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-db\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.991324 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991271 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.991324 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991285 2574 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-grpc-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.991324 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991298 2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.991324 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991305 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179"} err="failed to get container status \"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179\": rpc error: code = NotFound desc = could not find container \"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179\": container with ID starting with a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179 not found: ID does not exist" Jul 13 19:01:10.991324 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991313 2574 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.991324 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991319 2574 scope.go:117] "RemoveContainer" containerID="03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec" Jul 13 19:01:10.991518 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991326 2574 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-configmap-metrics-client-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.991518 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991335 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.991518 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991345 2574 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-metrics-client-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.991518 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991353 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 19:01:10.991518 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991358 2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e1e1de81-a666-4de5-976f-5ac4a6ec9235-config-out\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.991518 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991474 2574 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.991518 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991488 2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.991518 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991497 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e1e1de81-a666-4de5-976f-5ac4a6ec9235-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.991518 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991507 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khxqd\" (UniqueName: \"kubernetes.io/projected/e1e1de81-a666-4de5-976f-5ac4a6ec9235-kube-api-access-khxqd\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.991518 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991517 2574 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-kube-rbac-proxy\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.991842 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991531 2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e1e1de81-a666-4de5-976f-5ac4a6ec9235-tls-assets\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.991842 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991544 2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.991842 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991546 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec"} err="failed to get container status \"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec\": rpc error: code = NotFound desc = could not find container \"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec\": container with ID starting with 03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec not found: ID does not exist" Jul 13 19:01:10.991842 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991564 2574 scope.go:117] "RemoveContainer" containerID="0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd" Jul 13 19:01:10.991842 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991557 2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-web-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.991842 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991613 2574 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e1e1de81-a666-4de5-976f-5ac4a6ec9235-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:01:10.991842 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991748 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="kube-rbac-proxy" Jul 13 19:01:10.991842 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991763 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="kube-rbac-proxy" Jul 13 19:01:10.991842 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991794 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="prometheus" Jul 13 19:01:10.991842 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991801 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="prometheus" Jul 13 19:01:10.991842 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991799 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd"} err="failed to get container status \"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd\": rpc error: code = NotFound desc = could not find container \"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd\": container with ID starting with 0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd not found: ID does not exist" Jul 13 19:01:10.991842 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991809 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="config-reloader" Jul 13 19:01:10.991842 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991815 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="config-reloader" Jul 13 19:01:10.991842 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991825 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="kube-rbac-proxy-web" Jul 13 19:01:10.991842 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991831 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="kube-rbac-proxy-web" Jul 13 19:01:10.991842 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991838 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="init-config-reloader" Jul 13 19:01:10.991842 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991845 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="init-config-reloader" Jul 13 19:01:10.992427 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991855 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="kube-rbac-proxy-thanos" Jul 13 19:01:10.992427 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991860 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="kube-rbac-proxy-thanos" Jul 13 19:01:10.992427 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991873 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="thanos-sidecar" Jul 13 19:01:10.992427 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991878 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="thanos-sidecar" Jul 13 19:01:10.992427 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991814 2574 scope.go:117] "RemoveContainer" containerID="c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1" Jul 13 19:01:10.992427 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991956 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="thanos-sidecar" Jul 13 19:01:10.992427 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991964 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="prometheus" Jul 13 19:01:10.992427 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991973 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="kube-rbac-proxy-web" Jul 13 19:01:10.992427 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991982 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="kube-rbac-proxy" Jul 13 19:01:10.992427 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.991991 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="config-reloader" Jul 13 19:01:10.992427 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.992002 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" containerName="kube-rbac-proxy-thanos" Jul 13 19:01:10.992427 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.992105 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1"} err="failed to get container status \"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1\": rpc error: code = NotFound desc = could not find container \"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1\": container with ID starting with c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1 not found: ID does not exist" Jul 13 19:01:10.992427 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.992124 2574 scope.go:117] "RemoveContainer" containerID="e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26" Jul 13 19:01:10.992427 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.992347 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26"} err="failed to get container status \"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26\": rpc error: code = NotFound desc = could not find container \"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26\": container with ID starting with e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26 not found: ID does not exist" Jul 13 19:01:10.992427 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.992366 2574 scope.go:117] "RemoveContainer" containerID="47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4" Jul 13 19:01:10.993017 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.992617 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4"} err="failed to get container status \"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4\": rpc error: code = NotFound desc = could not find container \"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4\": container with ID starting with 47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4 not found: ID does not exist" Jul 13 19:01:10.993017 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.992642 2574 scope.go:117] "RemoveContainer" containerID="4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9" Jul 13 19:01:10.993017 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.992864 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9"} err="failed to get container status \"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9\": rpc error: code = NotFound desc = could not find container \"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9\": container with ID starting with 4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9 not found: ID does not exist" Jul 13 19:01:10.993017 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.992882 2574 scope.go:117] "RemoveContainer" containerID="a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179" Jul 13 19:01:10.993198 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.993070 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179"} err="failed to get container status \"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179\": rpc error: code = NotFound desc = could not find container \"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179\": container with ID starting with a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179 not found: ID does not exist" Jul 13 19:01:10.993198 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.993092 2574 scope.go:117] "RemoveContainer" containerID="03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec" Jul 13 19:01:10.993335 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.993318 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec"} err="failed to get container status \"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec\": rpc error: code = NotFound desc = could not find container \"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec\": container with ID starting with 03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec not found: ID does not exist" Jul 13 19:01:10.993394 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.993337 2574 scope.go:117] "RemoveContainer" containerID="0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd" Jul 13 19:01:10.993578 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.993561 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd"} err="failed to get container status \"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd\": rpc error: code = NotFound desc = could not find container \"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd\": container with ID starting with 0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd not found: ID does not exist" Jul 13 19:01:10.993642 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.993579 2574 scope.go:117] "RemoveContainer" containerID="c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1" Jul 13 19:01:10.993778 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.993761 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1"} err="failed to get container status \"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1\": rpc error: code = NotFound desc = could not find container \"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1\": container with ID starting with c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1 not found: ID does not exist" Jul 13 19:01:10.993778 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.993795 2574 scope.go:117] "RemoveContainer" containerID="e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26" Jul 13 19:01:10.994025 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.994007 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26"} err="failed to get container status \"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26\": rpc error: code = NotFound desc = could not find container \"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26\": container with ID starting with e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26 not found: ID does not exist" Jul 13 19:01:10.994095 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.994026 2574 scope.go:117] "RemoveContainer" containerID="47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4" Jul 13 19:01:10.994288 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.994270 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4"} err="failed to get container status \"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4\": rpc error: code = NotFound desc = could not find container \"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4\": container with ID starting with 47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4 not found: ID does not exist" Jul 13 19:01:10.994371 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.994289 2574 scope.go:117] "RemoveContainer" containerID="4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9" Jul 13 19:01:10.994504 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.994486 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9"} err="failed to get container status \"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9\": rpc error: code = NotFound desc = could not find container \"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9\": container with ID starting with 4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9 not found: ID does not exist" Jul 13 19:01:10.994569 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.994507 2574 scope.go:117] "RemoveContainer" containerID="a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179" Jul 13 19:01:10.994737 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.994719 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179"} err="failed to get container status \"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179\": rpc error: code = NotFound desc = could not find container \"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179\": container with ID starting with a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179 not found: ID does not exist" Jul 13 19:01:10.994826 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.994738 2574 scope.go:117] "RemoveContainer" containerID="03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec" Jul 13 19:01:10.994988 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.994970 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec"} err="failed to get container status \"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec\": rpc error: code = NotFound desc = could not find container \"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec\": container with ID starting with 03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec not found: ID does not exist" Jul 13 19:01:10.995055 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.994989 2574 scope.go:117] "RemoveContainer" containerID="0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd" Jul 13 19:01:10.995345 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.995266 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd"} err="failed to get container status \"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd\": rpc error: code = NotFound desc = could not find container \"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd\": container with ID starting with 0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd not found: ID does not exist" Jul 13 19:01:10.995345 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.995292 2574 scope.go:117] "RemoveContainer" containerID="c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1" Jul 13 19:01:10.995771 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.995710 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1"} err="failed to get container status \"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1\": rpc error: code = NotFound desc = could not find container \"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1\": container with ID starting with c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1 not found: ID does not exist" Jul 13 19:01:10.995771 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.995764 2574 scope.go:117] "RemoveContainer" containerID="e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26" Jul 13 19:01:10.995959 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.995848 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:10.996491 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.996467 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26"} err="failed to get container status \"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26\": rpc error: code = NotFound desc = could not find container \"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26\": container with ID starting with e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26 not found: ID does not exist" Jul 13 19:01:10.996491 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.996490 2574 scope.go:117] "RemoveContainer" containerID="47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4" Jul 13 19:01:10.996774 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.996754 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4"} err="failed to get container status \"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4\": rpc error: code = NotFound desc = could not find container \"47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4\": container with ID starting with 47f1abe2f4f04833c09a0cc71a4103b0230bc36322796974463a313bddbb6ce4 not found: ID does not exist" Jul 13 19:01:10.996859 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.996776 2574 scope.go:117] "RemoveContainer" containerID="4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9" Jul 13 19:01:10.997108 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.997087 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9"} err="failed to get container status \"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9\": rpc error: code = NotFound desc = could not find container \"4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9\": container with ID starting with 4a98f671261a15e5ca2922991690fbcbb91f1e64b1028080bde29f955920e6d9 not found: ID does not exist" Jul 13 19:01:10.997108 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.997107 2574 scope.go:117] "RemoveContainer" containerID="a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179" Jul 13 19:01:10.997353 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.997323 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179"} err="failed to get container status \"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179\": rpc error: code = NotFound desc = could not find container \"a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179\": container with ID starting with a16d8b062fada735151cf2c0d242a496bc57352f440291cbbba0a0cc8cd36179 not found: ID does not exist" Jul 13 19:01:10.997353 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.997343 2574 scope.go:117] "RemoveContainer" containerID="03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec" Jul 13 19:01:10.997558 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.997540 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec"} err="failed to get container status \"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec\": rpc error: code = NotFound desc = could not find container \"03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec\": container with ID starting with 03259eb9dbdb2d9c4aae4f0fd158630236a23ec3b991cb18a11cffa536cb3aec not found: ID does not exist" Jul 13 19:01:10.997627 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.997560 2574 scope.go:117] "RemoveContainer" containerID="0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd" Jul 13 19:01:10.997869 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.997840 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd"} err="failed to get container status \"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd\": rpc error: code = NotFound desc = could not find container \"0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd\": container with ID starting with 0a1b901de71b04e5398b82d8787288ce1ef1bf864cdbc3aab3926843120327bd not found: ID does not exist" Jul 13 19:01:10.997958 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.997871 2574 scope.go:117] "RemoveContainer" containerID="c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1" Jul 13 19:01:10.998149 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.998114 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1"} err="failed to get container status \"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1\": rpc error: code = NotFound desc = could not find container \"c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1\": container with ID starting with c97d0414d2f6175174a125c5b8a2adabad792fba8c0494911c65a7d76d77cbf1 not found: ID does not exist" Jul 13 19:01:10.998149 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.998148 2574 scope.go:117] "RemoveContainer" containerID="e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26" Jul 13 19:01:10.998276 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.998207 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 13 19:01:10.998276 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.998207 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 13 19:01:10.998423 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.998408 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 13 19:01:10.998507 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.998469 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cjm6n0v6b8e81\"" Jul 13 19:01:10.998507 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.998482 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 13 19:01:10.998610 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.998515 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26"} err="failed to get container status \"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26\": rpc error: code = NotFound desc = could not find container \"e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26\": container with ID starting with e85caaf5f028281f561010fbe3490cd017dd7853db72fa4fa2314957d00c8d26 not found: ID does not exist" Jul 13 19:01:10.998744 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.998728 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 13 19:01:10.998861 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.998831 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 13 19:01:10.998959 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.998910 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 13 19:01:10.998959 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.998956 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 13 19:01:10.999074 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.998980 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zkhw5\"" Jul 13 19:01:11.000548 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:10.999624 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 13 19:01:11.000548 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.000102 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 13 19:01:11.000548 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.000110 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 13 19:01:11.002159 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.001342 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 13 19:01:11.008077 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.008055 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 13 19:01:11.008870 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.008852 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 13 19:01:11.012796 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.012761 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 19:01:11.054163 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.054127 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1e1de81-a666-4de5-976f-5ac4a6ec9235" path="/var/lib/kubelet/pods/e1e1de81-a666-4de5-976f-5ac4a6ec9235/volumes" Jul 13 19:01:11.092103 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092069 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.092103 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092104 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.092306 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092136 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.092306 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092159 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.092306 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092182 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwrfq\" (UniqueName: \"kubernetes.io/projected/5e796353-dbba-44d4-a616-bd82bfd48cf3-kube-api-access-vwrfq\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.092306 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092277 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.092485 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092311 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.092485 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092334 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.092485 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092410 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.092485 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092451 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.092485 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092478 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5e796353-dbba-44d4-a616-bd82bfd48cf3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.092719 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092521 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-web-config\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.092719 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092540 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-config\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.092719 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092563 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5e796353-dbba-44d4-a616-bd82bfd48cf3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.092719 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092699 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.092917 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092750 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.092917 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092854 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5e796353-dbba-44d4-a616-bd82bfd48cf3-config-out\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.092917 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092902 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.093067 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092923 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.093067 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.092947 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.193921 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.193884 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-web-config\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.193921 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.193923 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-config\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.194136 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.193951 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5e796353-dbba-44d4-a616-bd82bfd48cf3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.194136 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.193979 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.194136 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.194085 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.194136 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.194105 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5e796353-dbba-44d4-a616-bd82bfd48cf3-config-out\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.194336 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.194146 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.194336 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.194166 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.194442 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.194347 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.194507 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.194451 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.194507 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.194480 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.194601 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.194510 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.194601 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.194533 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.194601 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.194561 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwrfq\" (UniqueName: \"kubernetes.io/projected/5e796353-dbba-44d4-a616-bd82bfd48cf3-kube-api-access-vwrfq\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.194748 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.194610 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.194748 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.194646 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.194748 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.194673 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.194748 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.194708 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.194748 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.194739 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.195023 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.194764 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5e796353-dbba-44d4-a616-bd82bfd48cf3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.195023 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.194950 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.195292 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.195143 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5e796353-dbba-44d4-a616-bd82bfd48cf3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.195292 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.195198 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.197497 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.197138 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5e796353-dbba-44d4-a616-bd82bfd48cf3-config-out\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.197497 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.197279 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-web-config\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.197497 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.197405 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5e796353-dbba-44d4-a616-bd82bfd48cf3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.197497 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.197433 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-config\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.198590 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.197525 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.198590 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.197797 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.198590 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.198046 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.198590 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.198102 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.198590 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.198184 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.198590 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.198235 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.198937 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.198655 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.199738 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.199716 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.199963 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.199940 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5e796353-dbba-44d4-a616-bd82bfd48cf3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.200013 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.199992 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.200069 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.200052 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.200242 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.200227 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5e796353-dbba-44d4-a616-bd82bfd48cf3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.204152 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.204131 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwrfq\" (UniqueName: \"kubernetes.io/projected/5e796353-dbba-44d4-a616-bd82bfd48cf3-kube-api-access-vwrfq\") pod \"prometheus-k8s-0\" (UID: \"5e796353-dbba-44d4-a616-bd82bfd48cf3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.311403 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.311363 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:11.450867 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.450840 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 19:01:11.453257 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:01:11.453230 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e796353_dbba_44d4_a616_bd82bfd48cf3.slice/crio-642026eb849c62772b09dbe821cfda14577f496dbdd291037086ca90c5a25216 WatchSource:0}: Error finding container 642026eb849c62772b09dbe821cfda14577f496dbdd291037086ca90c5a25216: Status 404 returned error can't find the container with id 642026eb849c62772b09dbe821cfda14577f496dbdd291037086ca90c5a25216 Jul 13 19:01:11.942384 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.942348 2574 generic.go:356] "Generic (PLEG): container finished" podID="5e796353-dbba-44d4-a616-bd82bfd48cf3" containerID="abb5ac35e04ffb763babf98794081d1ec32eaf2c54fd92c7749544f08eb09052" exitCode=0 Jul 13 19:01:11.942730 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.942396 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5e796353-dbba-44d4-a616-bd82bfd48cf3","Type":"ContainerDied","Data":"abb5ac35e04ffb763babf98794081d1ec32eaf2c54fd92c7749544f08eb09052"} Jul 13 19:01:11.942730 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:11.942459 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5e796353-dbba-44d4-a616-bd82bfd48cf3","Type":"ContainerStarted","Data":"642026eb849c62772b09dbe821cfda14577f496dbdd291037086ca90c5a25216"} Jul 13 19:01:12.950073 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:12.950033 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5e796353-dbba-44d4-a616-bd82bfd48cf3","Type":"ContainerStarted","Data":"5b07f1233aff826d95b7a4f059be9924e365718666bd3040b314d8535fb83e9d"} Jul 13 19:01:12.950510 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:12.950147 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5e796353-dbba-44d4-a616-bd82bfd48cf3","Type":"ContainerStarted","Data":"b91bd80ca0d9d7c6211a4abfb0cdaa1afabc40f04e5c38071b9a4bb09690423d"} Jul 13 19:01:12.950510 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:12.950196 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5e796353-dbba-44d4-a616-bd82bfd48cf3","Type":"ContainerStarted","Data":"6db7011a2869bbb75030bdd4d0ada1da30e9c8bf4d23d93a45577a29f440abc8"} Jul 13 19:01:12.950510 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:12.950237 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5e796353-dbba-44d4-a616-bd82bfd48cf3","Type":"ContainerStarted","Data":"8f66415571ec40de5c714562d2f8bdf03e1412f942be2bf133b5171c115d3e69"} Jul 13 19:01:12.950510 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:12.950273 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5e796353-dbba-44d4-a616-bd82bfd48cf3","Type":"ContainerStarted","Data":"3bd390b45e278a92e93cb14e32ccc7bc3fa8627646d2296131acae4cb6753878"} Jul 13 19:01:12.950510 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:12.950314 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5e796353-dbba-44d4-a616-bd82bfd48cf3","Type":"ContainerStarted","Data":"85d2dee1b77ebe906ad0316ce004579c0f8cbd06a17318a2d8fb4eda2c57fdcb"} Jul 13 19:01:12.980954 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:12.980892 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.960005041 podStartE2EDuration="2.980867172s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.862131ms" isStatefulPod=false podCreationTimestamp="2026-07-13 19:01:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:01:12.9782524 +0000 UTC m=+158.454706598" watchObservedRunningTime="2026-07-13 19:01:12.980867172 +0000 UTC m=+158.457321347" Jul 13 19:01:13.796004 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:13.795972 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:13.850433 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:13.850405 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:16.311760 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:16.311709 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:01:20.918276 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:20.917885 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:21.016151 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:21.016117 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:29.849001 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:29.848912 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:35.540436 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:35.540387 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:35.601291 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:35.601256 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:35.848436 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:35.848351 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:35.910373 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:35.910340 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:36.644841 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:36.644767 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:46.725985 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:46.725949 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:46.781136 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:46.781104 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:46.831860 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:46.831828 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:49.190772 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:49.190724 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:49.245964 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:49.245900 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:01:51.054626 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:01:51.054589 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:05.827666 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:05.827622 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:11.133174 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:11.133137 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:11.230346 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:11.230315 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:11.312421 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:11.312391 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:02:11.328315 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:11.328290 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:02:12.216423 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:12.216388 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:02:37.778324 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:37.778258 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:38.615556 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:38.615490 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:41.820416 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:41.820374 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:41.917624 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:41.917588 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:41.981746 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:41.981715 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:43.179631 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:43.179600 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:43.280684 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:43.280653 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:43.574674 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:43.574644 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:43.684733 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:43.684698 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:44.625175 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:44.625141 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cf55t"] Jul 13 19:02:44.628305 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:44.628288 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-cf55t" Jul 13 19:02:44.630664 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:44.630644 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 13 19:02:44.630775 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:44.630661 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 13 19:02:44.630775 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:44.630645 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-np5n6\"" Jul 13 19:02:44.637057 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:44.637034 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cf55t"] Jul 13 19:02:44.723013 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:44.722979 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c0d76f7-42b7-4ff5-9b60-46b82fb4b5cb-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cf55t\" (UID: \"8c0d76f7-42b7-4ff5-9b60-46b82fb4b5cb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cf55t" Jul 13 19:02:44.723013 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:44.723020 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l699j\" (UniqueName: \"kubernetes.io/projected/8c0d76f7-42b7-4ff5-9b60-46b82fb4b5cb-kube-api-access-l699j\") pod \"cert-manager-webhook-6486ffd997-cf55t\" (UID: \"8c0d76f7-42b7-4ff5-9b60-46b82fb4b5cb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cf55t" Jul 13 19:02:44.824371 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:44.824334 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c0d76f7-42b7-4ff5-9b60-46b82fb4b5cb-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cf55t\" (UID: \"8c0d76f7-42b7-4ff5-9b60-46b82fb4b5cb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cf55t" Jul 13 19:02:44.824371 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:44.824370 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l699j\" (UniqueName: \"kubernetes.io/projected/8c0d76f7-42b7-4ff5-9b60-46b82fb4b5cb-kube-api-access-l699j\") pod \"cert-manager-webhook-6486ffd997-cf55t\" (UID: \"8c0d76f7-42b7-4ff5-9b60-46b82fb4b5cb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cf55t" Jul 13 19:02:44.837350 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:44.837315 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c0d76f7-42b7-4ff5-9b60-46b82fb4b5cb-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cf55t\" (UID: \"8c0d76f7-42b7-4ff5-9b60-46b82fb4b5cb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cf55t" Jul 13 19:02:44.837473 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:44.837390 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l699j\" (UniqueName: \"kubernetes.io/projected/8c0d76f7-42b7-4ff5-9b60-46b82fb4b5cb-kube-api-access-l699j\") pod \"cert-manager-webhook-6486ffd997-cf55t\" (UID: \"8c0d76f7-42b7-4ff5-9b60-46b82fb4b5cb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cf55t" Jul 13 19:02:44.957703 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:44.957617 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-cf55t" Jul 13 19:02:45.090198 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:45.089971 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cf55t"] Jul 13 19:02:45.094114 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:02:45.094086 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c0d76f7_42b7_4ff5_9b60_46b82fb4b5cb.slice/crio-1adf98f329b2a368929ce1ee0ba6b3f756a63c6ddd3195efb2780a4a8f0ab1ba WatchSource:0}: Error finding container 1adf98f329b2a368929ce1ee0ba6b3f756a63c6ddd3195efb2780a4a8f0ab1ba: Status 404 returned error can't find the container with id 1adf98f329b2a368929ce1ee0ba6b3f756a63c6ddd3195efb2780a4a8f0ab1ba Jul 13 19:02:45.343679 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:45.343641 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-cf55t" event={"ID":"8c0d76f7-42b7-4ff5-9b60-46b82fb4b5cb","Type":"ContainerStarted","Data":"1adf98f329b2a368929ce1ee0ba6b3f756a63c6ddd3195efb2780a4a8f0ab1ba"} Jul 13 19:02:46.818179 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:46.818120 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:46.884250 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:46.884191 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:48.529054 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:48.529004 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:48.974506 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:48.974472 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:50.369475 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:50.369433 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-cf55t" event={"ID":"8c0d76f7-42b7-4ff5-9b60-46b82fb4b5cb","Type":"ContainerStarted","Data":"22be371a4754578031bedc0e1d44aa016caf42103bdd4bb9ab11bf86ab0041a9"} Jul 13 19:02:50.369875 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:50.369552 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-cf55t" Jul 13 19:02:50.388683 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:50.388632 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-cf55t" podStartSLOduration=2.005089117 podStartE2EDuration="6.388617852s" totalImagesPullingTime="4.383528735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:02:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:02:50.386610433 +0000 UTC m=+255.863064609" watchObservedRunningTime="2026-07-13 19:02:50.388617852 +0000 UTC m=+255.865072025" Jul 13 19:02:51.638222 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:51.638182 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:51.728895 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:51.728856 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:53.544990 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:53.544953 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:55.853095 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:55.853060 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:55.961530 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:55.961494 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:02:56.381923 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:56.381891 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-cf55t" Jul 13 19:02:57.695646 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:57.695612 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qznmt"] Jul 13 19:02:57.698702 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:57.698684 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qznmt" Jul 13 19:02:57.701044 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:57.701023 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-jbmcm\"" Jul 13 19:02:57.701740 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:57.701722 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 13 19:02:57.701836 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:57.701749 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 13 19:02:57.707963 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:57.707942 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qznmt"] Jul 13 19:02:57.738131 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:57.738103 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46c90dbb-e080-4d9d-bad5-bb0bd08c7b6b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qznmt\" (UID: \"46c90dbb-e080-4d9d-bad5-bb0bd08c7b6b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qznmt" Jul 13 19:02:57.738271 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:57.738165 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg5rw\" (UniqueName: \"kubernetes.io/projected/46c90dbb-e080-4d9d-bad5-bb0bd08c7b6b-kube-api-access-mg5rw\") pod \"openshift-lws-operator-fd8ccff4c-qznmt\" (UID: \"46c90dbb-e080-4d9d-bad5-bb0bd08c7b6b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qznmt" Jul 13 19:02:57.838930 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:57.838894 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46c90dbb-e080-4d9d-bad5-bb0bd08c7b6b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qznmt\" (UID: \"46c90dbb-e080-4d9d-bad5-bb0bd08c7b6b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qznmt" Jul 13 19:02:57.839129 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:57.838949 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg5rw\" (UniqueName: \"kubernetes.io/projected/46c90dbb-e080-4d9d-bad5-bb0bd08c7b6b-kube-api-access-mg5rw\") pod \"openshift-lws-operator-fd8ccff4c-qznmt\" (UID: \"46c90dbb-e080-4d9d-bad5-bb0bd08c7b6b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qznmt" Jul 13 19:02:57.839335 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:57.839311 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46c90dbb-e080-4d9d-bad5-bb0bd08c7b6b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qznmt\" (UID: \"46c90dbb-e080-4d9d-bad5-bb0bd08c7b6b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qznmt" Jul 13 19:02:57.847074 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:57.847054 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg5rw\" (UniqueName: \"kubernetes.io/projected/46c90dbb-e080-4d9d-bad5-bb0bd08c7b6b-kube-api-access-mg5rw\") pod \"openshift-lws-operator-fd8ccff4c-qznmt\" (UID: \"46c90dbb-e080-4d9d-bad5-bb0bd08c7b6b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qznmt" Jul 13 19:02:58.008465 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:58.008381 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qznmt" Jul 13 19:02:58.135796 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:58.135761 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qznmt"] Jul 13 19:02:58.138445 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:02:58.138421 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46c90dbb_e080_4d9d_bad5_bb0bd08c7b6b.slice/crio-4cf92feb81d88b7bb2b7b68bb195839be190cf0adc100c5fa6b31f4d88321612 WatchSource:0}: Error finding container 4cf92feb81d88b7bb2b7b68bb195839be190cf0adc100c5fa6b31f4d88321612: Status 404 returned error can't find the container with id 4cf92feb81d88b7bb2b7b68bb195839be190cf0adc100c5fa6b31f4d88321612 Jul 13 19:02:58.411846 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:02:58.411811 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qznmt" event={"ID":"46c90dbb-e080-4d9d-bad5-bb0bd08c7b6b","Type":"ContainerStarted","Data":"4cf92feb81d88b7bb2b7b68bb195839be190cf0adc100c5fa6b31f4d88321612"} Jul 13 19:03:01.427762 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:01.427728 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qznmt" event={"ID":"46c90dbb-e080-4d9d-bad5-bb0bd08c7b6b","Type":"ContainerStarted","Data":"18c13bdb34d3365420c460e7420c7d1cd8a394e51c90b15d69678c833e66ef60"} Jul 13 19:03:01.444010 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:01.443956 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qznmt" podStartSLOduration=2.06202393 podStartE2EDuration="4.443939698s" totalImagesPullingTime="2.381915768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:02:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:03:01.442334968 +0000 UTC m=+266.918789203" watchObservedRunningTime="2026-07-13 19:03:01.443939698 +0000 UTC m=+266.920393873" Jul 13 19:03:02.182643 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:02.182605 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:02.283186 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:02.283154 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:04.240972 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:04.240942 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:18.624161 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.624117 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:18.729101 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.729058 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p"] Jul 13 19:03:18.733544 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.733528 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" Jul 13 19:03:18.740888 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.740865 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 13 19:03:18.741002 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.740885 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-c4h5v\"" Jul 13 19:03:18.741002 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.740898 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 13 19:03:18.741002 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.740865 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 13 19:03:18.770421 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.770395 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p"] Jul 13 19:03:18.775970 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.775943 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:18.813397 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.813361 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a96a66d-188a-4dc6-abe7-cba90e7d793a-cert\") pod \"lws-controller-manager-64bdc88f66-2dc2p\" (UID: \"8a96a66d-188a-4dc6-abe7-cba90e7d793a\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" Jul 13 19:03:18.813578 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.813407 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjxwq\" (UniqueName: \"kubernetes.io/projected/8a96a66d-188a-4dc6-abe7-cba90e7d793a-kube-api-access-sjxwq\") pod \"lws-controller-manager-64bdc88f66-2dc2p\" (UID: \"8a96a66d-188a-4dc6-abe7-cba90e7d793a\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" Jul 13 19:03:18.813578 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.813518 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8a96a66d-188a-4dc6-abe7-cba90e7d793a-manager-config\") pod \"lws-controller-manager-64bdc88f66-2dc2p\" (UID: \"8a96a66d-188a-4dc6-abe7-cba90e7d793a\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" Jul 13 19:03:18.813665 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.813632 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a96a66d-188a-4dc6-abe7-cba90e7d793a-metrics-cert\") pod \"lws-controller-manager-64bdc88f66-2dc2p\" (UID: \"8a96a66d-188a-4dc6-abe7-cba90e7d793a\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" Jul 13 19:03:18.914573 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.914481 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a96a66d-188a-4dc6-abe7-cba90e7d793a-metrics-cert\") pod \"lws-controller-manager-64bdc88f66-2dc2p\" (UID: \"8a96a66d-188a-4dc6-abe7-cba90e7d793a\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" Jul 13 19:03:18.914573 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.914542 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a96a66d-188a-4dc6-abe7-cba90e7d793a-cert\") pod \"lws-controller-manager-64bdc88f66-2dc2p\" (UID: \"8a96a66d-188a-4dc6-abe7-cba90e7d793a\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" Jul 13 19:03:18.914832 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.914577 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjxwq\" (UniqueName: \"kubernetes.io/projected/8a96a66d-188a-4dc6-abe7-cba90e7d793a-kube-api-access-sjxwq\") pod \"lws-controller-manager-64bdc88f66-2dc2p\" (UID: \"8a96a66d-188a-4dc6-abe7-cba90e7d793a\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" Jul 13 19:03:18.914832 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.914627 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8a96a66d-188a-4dc6-abe7-cba90e7d793a-manager-config\") pod \"lws-controller-manager-64bdc88f66-2dc2p\" (UID: \"8a96a66d-188a-4dc6-abe7-cba90e7d793a\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" Jul 13 19:03:18.915395 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.915373 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8a96a66d-188a-4dc6-abe7-cba90e7d793a-manager-config\") pod \"lws-controller-manager-64bdc88f66-2dc2p\" (UID: \"8a96a66d-188a-4dc6-abe7-cba90e7d793a\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" Jul 13 19:03:18.917038 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.917014 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a96a66d-188a-4dc6-abe7-cba90e7d793a-cert\") pod \"lws-controller-manager-64bdc88f66-2dc2p\" (UID: \"8a96a66d-188a-4dc6-abe7-cba90e7d793a\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" Jul 13 19:03:18.917134 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.917060 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a96a66d-188a-4dc6-abe7-cba90e7d793a-metrics-cert\") pod \"lws-controller-manager-64bdc88f66-2dc2p\" (UID: \"8a96a66d-188a-4dc6-abe7-cba90e7d793a\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" Jul 13 19:03:18.927555 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:18.927527 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjxwq\" (UniqueName: \"kubernetes.io/projected/8a96a66d-188a-4dc6-abe7-cba90e7d793a-kube-api-access-sjxwq\") pod \"lws-controller-manager-64bdc88f66-2dc2p\" (UID: \"8a96a66d-188a-4dc6-abe7-cba90e7d793a\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" Jul 13 19:03:19.042861 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:19.042825 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" Jul 13 19:03:19.084391 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:19.084356 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:19.190592 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:19.190571 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p"] Jul 13 19:03:19.192934 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:03:19.192899 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a96a66d_188a_4dc6_abe7_cba90e7d793a.slice/crio-215d832988fadfdaefdcde6bece9b99660b833b00e19695e9bb8fed0dc7d6ed8 WatchSource:0}: Error finding container 215d832988fadfdaefdcde6bece9b99660b833b00e19695e9bb8fed0dc7d6ed8: Status 404 returned error can't find the container with id 215d832988fadfdaefdcde6bece9b99660b833b00e19695e9bb8fed0dc7d6ed8 Jul 13 19:03:19.219483 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:19.219451 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:19.503406 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:19.503321 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" event={"ID":"8a96a66d-188a-4dc6-abe7-cba90e7d793a","Type":"ContainerStarted","Data":"215d832988fadfdaefdcde6bece9b99660b833b00e19695e9bb8fed0dc7d6ed8"} Jul 13 19:03:19.697371 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:19.697333 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:20.100739 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.100701 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh"] Jul 13 19:03:20.104044 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.104028 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:20.109768 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.109739 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 13 19:03:20.109906 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.109822 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 13 19:03:20.110409 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.110392 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q7qnq\"" Jul 13 19:03:20.110665 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.110651 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 13 19:03:20.111966 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.111937 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 13 19:03:20.150196 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.150160 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh"] Jul 13 19:03:20.230529 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.230493 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/491181a6-8b08-403d-bb53-ddab58861a6a-webhook-cert\") pod \"opendatahub-operator-controller-manager-559f5498f-l89sh\" (UID: \"491181a6-8b08-403d-bb53-ddab58861a6a\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:20.230700 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.230541 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/491181a6-8b08-403d-bb53-ddab58861a6a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-559f5498f-l89sh\" (UID: \"491181a6-8b08-403d-bb53-ddab58861a6a\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:20.230700 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.230626 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/491181a6-8b08-403d-bb53-ddab58861a6a-tmp\") pod \"opendatahub-operator-controller-manager-559f5498f-l89sh\" (UID: \"491181a6-8b08-403d-bb53-ddab58861a6a\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:20.230700 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.230655 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/491181a6-8b08-403d-bb53-ddab58861a6a-manifests\") pod \"opendatahub-operator-controller-manager-559f5498f-l89sh\" (UID: \"491181a6-8b08-403d-bb53-ddab58861a6a\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:20.230900 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.230704 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4d5m\" (UniqueName: \"kubernetes.io/projected/491181a6-8b08-403d-bb53-ddab58861a6a-kube-api-access-s4d5m\") pod \"opendatahub-operator-controller-manager-559f5498f-l89sh\" (UID: \"491181a6-8b08-403d-bb53-ddab58861a6a\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:20.331246 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.331209 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/491181a6-8b08-403d-bb53-ddab58861a6a-tmp\") pod \"opendatahub-operator-controller-manager-559f5498f-l89sh\" (UID: \"491181a6-8b08-403d-bb53-ddab58861a6a\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:20.331246 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.331246 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/491181a6-8b08-403d-bb53-ddab58861a6a-manifests\") pod \"opendatahub-operator-controller-manager-559f5498f-l89sh\" (UID: \"491181a6-8b08-403d-bb53-ddab58861a6a\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:20.331485 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.331287 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4d5m\" (UniqueName: \"kubernetes.io/projected/491181a6-8b08-403d-bb53-ddab58861a6a-kube-api-access-s4d5m\") pod \"opendatahub-operator-controller-manager-559f5498f-l89sh\" (UID: \"491181a6-8b08-403d-bb53-ddab58861a6a\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:20.331485 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.331312 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/491181a6-8b08-403d-bb53-ddab58861a6a-webhook-cert\") pod \"opendatahub-operator-controller-manager-559f5498f-l89sh\" (UID: \"491181a6-8b08-403d-bb53-ddab58861a6a\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:20.331485 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.331346 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/491181a6-8b08-403d-bb53-ddab58861a6a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-559f5498f-l89sh\" (UID: \"491181a6-8b08-403d-bb53-ddab58861a6a\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:20.331729 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.331706 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/491181a6-8b08-403d-bb53-ddab58861a6a-manifests\") pod \"opendatahub-operator-controller-manager-559f5498f-l89sh\" (UID: \"491181a6-8b08-403d-bb53-ddab58861a6a\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:20.333530 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.333504 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/491181a6-8b08-403d-bb53-ddab58861a6a-tmp\") pod \"opendatahub-operator-controller-manager-559f5498f-l89sh\" (UID: \"491181a6-8b08-403d-bb53-ddab58861a6a\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:20.333751 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.333735 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/491181a6-8b08-403d-bb53-ddab58861a6a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-559f5498f-l89sh\" (UID: \"491181a6-8b08-403d-bb53-ddab58861a6a\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:20.333963 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.333940 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/491181a6-8b08-403d-bb53-ddab58861a6a-webhook-cert\") pod \"opendatahub-operator-controller-manager-559f5498f-l89sh\" (UID: \"491181a6-8b08-403d-bb53-ddab58861a6a\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:20.339487 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.339458 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4d5m\" (UniqueName: \"kubernetes.io/projected/491181a6-8b08-403d-bb53-ddab58861a6a-kube-api-access-s4d5m\") pod \"opendatahub-operator-controller-manager-559f5498f-l89sh\" (UID: \"491181a6-8b08-403d-bb53-ddab58861a6a\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:20.415650 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.415569 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:20.547735 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:20.547701 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh"] Jul 13 19:03:20.550854 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:03:20.550821 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod491181a6_8b08_403d_bb53_ddab58861a6a.slice/crio-11f9aad7ad544c82bd1d2076bd6b1c2498f996e54fb29d2c7184f294c24fe4c1 WatchSource:0}: Error finding container 11f9aad7ad544c82bd1d2076bd6b1c2498f996e54fb29d2c7184f294c24fe4c1: Status 404 returned error can't find the container with id 11f9aad7ad544c82bd1d2076bd6b1c2498f996e54fb29d2c7184f294c24fe4c1 Jul 13 19:03:21.517276 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:21.517233 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" event={"ID":"491181a6-8b08-403d-bb53-ddab58861a6a","Type":"ContainerStarted","Data":"11f9aad7ad544c82bd1d2076bd6b1c2498f996e54fb29d2c7184f294c24fe4c1"} Jul 13 19:03:22.125043 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:22.124992 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:25.538727 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:25.538688 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" event={"ID":"8a96a66d-188a-4dc6-abe7-cba90e7d793a","Type":"ContainerStarted","Data":"cdaae14328106320cc527253090adad53a952ac12aacd2494c4d047958fa75ec"} Jul 13 19:03:25.539197 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:25.538753 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" Jul 13 19:03:25.540324 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:25.540300 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" event={"ID":"491181a6-8b08-403d-bb53-ddab58861a6a","Type":"ContainerStarted","Data":"bf22f495e207084272c549ff8d01b9cae1ffe76995a646443df5a95fb67c0088"} Jul 13 19:03:25.554734 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:25.554681 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" podStartSLOduration=2.202367135 podStartE2EDuration="7.554664725s" totalImagesPullingTime="5.35229759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:03:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:03:25.553906663 +0000 UTC m=+291.030360838" watchObservedRunningTime="2026-07-13 19:03:25.554664725 +0000 UTC m=+291.031118899" Jul 13 19:03:26.546308 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:26.546271 2574 generic.go:356] "Generic (PLEG): container finished" podID="491181a6-8b08-403d-bb53-ddab58861a6a" containerID="bf22f495e207084272c549ff8d01b9cae1ffe76995a646443df5a95fb67c0088" exitCode=0 Jul 13 19:03:26.546830 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:26.546363 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" event={"ID":"491181a6-8b08-403d-bb53-ddab58861a6a","Type":"ContainerDied","Data":"bf22f495e207084272c549ff8d01b9cae1ffe76995a646443df5a95fb67c0088"} Jul 13 19:03:27.552885 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:27.552850 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" event={"ID":"491181a6-8b08-403d-bb53-ddab58861a6a","Type":"ContainerStarted","Data":"c531d0757b4f2bb75d24882a0ac6ca18cc805ee4977183fb64bb1fa62c9a0c33"} Jul 13 19:03:27.553300 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:27.552945 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:27.578443 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:27.578391 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" podStartSLOduration=2.088325075 podStartE2EDuration="7.578370818s" totalImagesPullingTime="4.377194903s" totalInitContainerRuntime="1.11285084s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:03:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:03:27.577093614 +0000 UTC m=+293.053547996" watchObservedRunningTime="2026-07-13 19:03:27.578370818 +0000 UTC m=+293.054824994" Jul 13 19:03:34.898982 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:34.898949 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 19:03:34.907424 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:34.907400 2574 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 19:03:34.908156 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:34.908135 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 19:03:36.548844 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:36.548812 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-2dc2p" Jul 13 19:03:38.559250 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:38.559217 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-l89sh" Jul 13 19:03:40.262225 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:40.262188 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:40.475132 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:40.475095 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:45.959857 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:45.959813 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:46.011055 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.011024 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-85c9d98b58-nzdqf"] Jul 13 19:03:46.013568 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.013547 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85c9d98b58-nzdqf" Jul 13 19:03:46.015895 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.015870 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 13 19:03:46.016019 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.015944 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8mhb8\"" Jul 13 19:03:46.016019 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.015967 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 13 19:03:46.016019 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.015980 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 13 19:03:46.016369 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.016094 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 13 19:03:46.028140 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.028116 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85c9d98b58-nzdqf"] Jul 13 19:03:46.068447 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.068401 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/820904a9-84c2-4516-8669-5b979c4349f0-tls-certs\") pod \"kube-auth-proxy-85c9d98b58-nzdqf\" (UID: \"820904a9-84c2-4516-8669-5b979c4349f0\") " pod="openshift-ingress/kube-auth-proxy-85c9d98b58-nzdqf" Jul 13 19:03:46.068447 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.068451 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvm77\" (UniqueName: \"kubernetes.io/projected/820904a9-84c2-4516-8669-5b979c4349f0-kube-api-access-kvm77\") pod \"kube-auth-proxy-85c9d98b58-nzdqf\" (UID: \"820904a9-84c2-4516-8669-5b979c4349f0\") " pod="openshift-ingress/kube-auth-proxy-85c9d98b58-nzdqf" Jul 13 19:03:46.068694 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.068547 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/820904a9-84c2-4516-8669-5b979c4349f0-tmp\") pod \"kube-auth-proxy-85c9d98b58-nzdqf\" (UID: \"820904a9-84c2-4516-8669-5b979c4349f0\") " pod="openshift-ingress/kube-auth-proxy-85c9d98b58-nzdqf" Jul 13 19:03:46.169438 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.169394 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/820904a9-84c2-4516-8669-5b979c4349f0-tmp\") pod \"kube-auth-proxy-85c9d98b58-nzdqf\" (UID: \"820904a9-84c2-4516-8669-5b979c4349f0\") " pod="openshift-ingress/kube-auth-proxy-85c9d98b58-nzdqf" Jul 13 19:03:46.169716 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.169463 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/820904a9-84c2-4516-8669-5b979c4349f0-tls-certs\") pod \"kube-auth-proxy-85c9d98b58-nzdqf\" (UID: \"820904a9-84c2-4516-8669-5b979c4349f0\") " pod="openshift-ingress/kube-auth-proxy-85c9d98b58-nzdqf" Jul 13 19:03:46.169716 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.169488 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvm77\" (UniqueName: \"kubernetes.io/projected/820904a9-84c2-4516-8669-5b979c4349f0-kube-api-access-kvm77\") pod \"kube-auth-proxy-85c9d98b58-nzdqf\" (UID: \"820904a9-84c2-4516-8669-5b979c4349f0\") " pod="openshift-ingress/kube-auth-proxy-85c9d98b58-nzdqf" Jul 13 19:03:46.169716 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:03:46.169631 2574 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 13 19:03:46.169716 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:03:46.169701 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/820904a9-84c2-4516-8669-5b979c4349f0-tls-certs podName:820904a9-84c2-4516-8669-5b979c4349f0 nodeName:}" failed. No retries permitted until 2026-07-13 19:03:46.669683322 +0000 UTC m=+312.146137474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/820904a9-84c2-4516-8669-5b979c4349f0-tls-certs") pod "kube-auth-proxy-85c9d98b58-nzdqf" (UID: "820904a9-84c2-4516-8669-5b979c4349f0") : secret "kube-auth-proxy-tls" not found Jul 13 19:03:46.171585 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.171555 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/820904a9-84c2-4516-8669-5b979c4349f0-tmp\") pod \"kube-auth-proxy-85c9d98b58-nzdqf\" (UID: \"820904a9-84c2-4516-8669-5b979c4349f0\") " pod="openshift-ingress/kube-auth-proxy-85c9d98b58-nzdqf" Jul 13 19:03:46.191759 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.191726 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvm77\" (UniqueName: \"kubernetes.io/projected/820904a9-84c2-4516-8669-5b979c4349f0-kube-api-access-kvm77\") pod \"kube-auth-proxy-85c9d98b58-nzdqf\" (UID: \"820904a9-84c2-4516-8669-5b979c4349f0\") " pod="openshift-ingress/kube-auth-proxy-85c9d98b58-nzdqf" Jul 13 19:03:46.301238 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.301201 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:46.399124 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.399091 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:46.500030 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.500000 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:46.674466 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.674383 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/820904a9-84c2-4516-8669-5b979c4349f0-tls-certs\") pod \"kube-auth-proxy-85c9d98b58-nzdqf\" (UID: \"820904a9-84c2-4516-8669-5b979c4349f0\") " pod="openshift-ingress/kube-auth-proxy-85c9d98b58-nzdqf" Jul 13 19:03:46.676849 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.676821 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/820904a9-84c2-4516-8669-5b979c4349f0-tls-certs\") pod \"kube-auth-proxy-85c9d98b58-nzdqf\" (UID: \"820904a9-84c2-4516-8669-5b979c4349f0\") " pod="openshift-ingress/kube-auth-proxy-85c9d98b58-nzdqf" Jul 13 19:03:46.925092 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.925001 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85c9d98b58-nzdqf" Jul 13 19:03:46.990216 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:46.990132 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:47.078031 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:47.077997 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85c9d98b58-nzdqf"] Jul 13 19:03:47.081254 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:03:47.081225 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod820904a9_84c2_4516_8669_5b979c4349f0.slice/crio-3430691d1041d76ffe74c873a3cb5ac928e9abbc70a017d3aa668dd3304e0bc2 WatchSource:0}: Error finding container 3430691d1041d76ffe74c873a3cb5ac928e9abbc70a017d3aa668dd3304e0bc2: Status 404 returned error can't find the container with id 3430691d1041d76ffe74c873a3cb5ac928e9abbc70a017d3aa668dd3304e0bc2 Jul 13 19:03:47.083043 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:47.083028 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 19:03:47.308064 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:47.308031 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:47.481802 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:47.481755 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:47.552499 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:47.552466 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:47.644492 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:47.644405 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85c9d98b58-nzdqf" event={"ID":"820904a9-84c2-4516-8669-5b979c4349f0","Type":"ContainerStarted","Data":"3430691d1041d76ffe74c873a3cb5ac928e9abbc70a017d3aa668dd3304e0bc2"} Jul 13 19:03:48.603243 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:48.603203 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:50.662854 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:50.662821 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85c9d98b58-nzdqf" event={"ID":"820904a9-84c2-4516-8669-5b979c4349f0","Type":"ContainerStarted","Data":"f3fac498f62d4d155268745ea901e94db465c8f928eecbb7433aa36e6ca5c29e"} Jul 13 19:03:50.679492 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:50.679441 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-85c9d98b58-nzdqf" podStartSLOduration=2.196453171 podStartE2EDuration="5.679425425s" totalImagesPullingTime="3.482972254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:03:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:03:50.677904053 +0000 UTC m=+316.154358227" watchObservedRunningTime="2026-07-13 19:03:50.679425425 +0000 UTC m=+316.155879598" Jul 13 19:03:53.589358 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:53.589328 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:53.696487 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:53.696452 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:53.963432 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:53.963341 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:54.258848 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:54.258632 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:54.352058 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:54.352023 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:58.581441 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:58.581404 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:58.731685 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:58.731618 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:58.846649 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:58.846545 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:59.018199 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:59.018163 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:03:59.156691 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:03:59.156603 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:04:41.776020 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:04:41.775934 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:04:41.877679 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:04:41.877640 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:04:42.006473 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:04:42.006438 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:04:42.693233 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:04:42.693200 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:04:42.903926 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:04:42.903897 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:06.354485 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:06.354446 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:08.227512 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:08.227483 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:08.295245 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:08.295215 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:12.070161 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.070108 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:12.169050 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.169019 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bc895d6b8-xfbss"] Jul 13 19:05:12.172568 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.172551 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.174859 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.174827 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 19:05:12.175384 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.175366 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 19:05:12.175745 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.175728 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 19:05:12.175844 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.175825 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 19:05:12.176145 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.176123 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 19:05:12.176553 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.176536 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jrnfq\"" Jul 13 19:05:12.182904 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.182886 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 19:05:12.189325 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.189302 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bc895d6b8-xfbss"] Jul 13 19:05:12.247576 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.247543 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:12.249625 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.249605 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6018c5dd-8193-49fa-acf5-2311dd79a62b-service-ca\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.249697 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.249639 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6018c5dd-8193-49fa-acf5-2311dd79a62b-console-serving-cert\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.249697 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.249666 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crsl9\" (UniqueName: \"kubernetes.io/projected/6018c5dd-8193-49fa-acf5-2311dd79a62b-kube-api-access-crsl9\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.249763 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.249730 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6018c5dd-8193-49fa-acf5-2311dd79a62b-console-oauth-config\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.249763 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.249759 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6018c5dd-8193-49fa-acf5-2311dd79a62b-oauth-serving-cert\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.249846 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.249776 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6018c5dd-8193-49fa-acf5-2311dd79a62b-trusted-ca-bundle\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.249846 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.249829 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6018c5dd-8193-49fa-acf5-2311dd79a62b-console-config\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.342314 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.342231 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:12.350455 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.350429 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6018c5dd-8193-49fa-acf5-2311dd79a62b-console-oauth-config\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.350563 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.350469 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6018c5dd-8193-49fa-acf5-2311dd79a62b-oauth-serving-cert\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.350563 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.350487 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6018c5dd-8193-49fa-acf5-2311dd79a62b-trusted-ca-bundle\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.350563 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.350518 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6018c5dd-8193-49fa-acf5-2311dd79a62b-console-config\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.350563 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.350558 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6018c5dd-8193-49fa-acf5-2311dd79a62b-service-ca\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.350766 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.350584 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6018c5dd-8193-49fa-acf5-2311dd79a62b-console-serving-cert\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.350766 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.350613 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crsl9\" (UniqueName: \"kubernetes.io/projected/6018c5dd-8193-49fa-acf5-2311dd79a62b-kube-api-access-crsl9\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.351413 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.351374 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6018c5dd-8193-49fa-acf5-2311dd79a62b-oauth-serving-cert\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.351413 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.351390 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6018c5dd-8193-49fa-acf5-2311dd79a62b-service-ca\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.351603 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.351436 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6018c5dd-8193-49fa-acf5-2311dd79a62b-console-config\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.351603 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.351443 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6018c5dd-8193-49fa-acf5-2311dd79a62b-trusted-ca-bundle\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.353056 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.353039 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6018c5dd-8193-49fa-acf5-2311dd79a62b-console-oauth-config\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.353122 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.353108 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6018c5dd-8193-49fa-acf5-2311dd79a62b-console-serving-cert\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.363725 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.363701 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crsl9\" (UniqueName: \"kubernetes.io/projected/6018c5dd-8193-49fa-acf5-2311dd79a62b-kube-api-access-crsl9\") pod \"console-6bc895d6b8-xfbss\" (UID: \"6018c5dd-8193-49fa-acf5-2311dd79a62b\") " pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.482182 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.482140 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:12.633390 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:12.633364 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bc895d6b8-xfbss"] Jul 13 19:05:12.635063 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:05:12.635031 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6018c5dd_8193_49fa_acf5_2311dd79a62b.slice/crio-d7277e87681a3accadc2e911b27ca8ec72acebbe9d3b4e210280a25d3fd1432b WatchSource:0}: Error finding container d7277e87681a3accadc2e911b27ca8ec72acebbe9d3b4e210280a25d3fd1432b: Status 404 returned error can't find the container with id d7277e87681a3accadc2e911b27ca8ec72acebbe9d3b4e210280a25d3fd1432b Jul 13 19:05:13.054373 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:13.054339 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bc895d6b8-xfbss" event={"ID":"6018c5dd-8193-49fa-acf5-2311dd79a62b","Type":"ContainerStarted","Data":"7871c34f9bd756b1eb134d42ebd25b05bba68071bbc2066463d14e83f995744a"} Jul 13 19:05:13.054559 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:13.054387 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bc895d6b8-xfbss" event={"ID":"6018c5dd-8193-49fa-acf5-2311dd79a62b","Type":"ContainerStarted","Data":"d7277e87681a3accadc2e911b27ca8ec72acebbe9d3b4e210280a25d3fd1432b"} Jul 13 19:05:13.078698 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:13.078652 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bc895d6b8-xfbss" podStartSLOduration=1.078637814 podStartE2EDuration="1.078637814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:05:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:05:13.076459174 +0000 UTC m=+398.552913343" watchObservedRunningTime="2026-07-13 19:05:13.078637814 +0000 UTC m=+398.555091988" Jul 13 19:05:15.091341 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:15.091297 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:15.848578 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:15.848551 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:18.203184 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:18.203150 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:19.896299 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:19.896266 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:20.679714 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:20.679678 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:22.482865 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:22.482826 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:22.483326 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:22.483154 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:22.487906 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:22.487886 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:23.104508 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:23.104478 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bc895d6b8-xfbss" Jul 13 19:05:24.067874 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:24.067840 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:24.168998 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:24.168965 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:31.707602 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:31.707562 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:31.834193 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:31.834152 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:36.870219 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:36.870178 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:36.971654 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:36.971619 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:37.592847 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:37.592812 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:37.705384 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:37.705348 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:40.231038 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:40.231006 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:40.746908 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:40.746873 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:41.262294 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:41.262187 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:41.367341 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:41.367298 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:46.775159 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:46.775126 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:50.322644 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:50.322608 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-574m7"] Jul 13 19:05:50.326422 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:50.326397 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-574m7" Jul 13 19:05:50.329840 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:50.329815 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 19:05:50.330410 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:50.330387 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 19:05:50.330580 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:50.330557 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-5766p\"" Jul 13 19:05:50.336957 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:50.336926 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-574m7"] Jul 13 19:05:50.397518 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:50.397475 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhzhq\" (UniqueName: \"kubernetes.io/projected/1135c060-7bcd-4879-8212-4527f428e844-kube-api-access-mhzhq\") pod \"limitador-operator-controller-manager-859d7b78bd-574m7\" (UID: \"1135c060-7bcd-4879-8212-4527f428e844\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-574m7" Jul 13 19:05:50.498479 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:50.498445 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhzhq\" (UniqueName: \"kubernetes.io/projected/1135c060-7bcd-4879-8212-4527f428e844-kube-api-access-mhzhq\") pod \"limitador-operator-controller-manager-859d7b78bd-574m7\" (UID: \"1135c060-7bcd-4879-8212-4527f428e844\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-574m7" Jul 13 19:05:50.506752 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:50.506719 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhzhq\" (UniqueName: \"kubernetes.io/projected/1135c060-7bcd-4879-8212-4527f428e844-kube-api-access-mhzhq\") pod \"limitador-operator-controller-manager-859d7b78bd-574m7\" (UID: \"1135c060-7bcd-4879-8212-4527f428e844\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-574m7" Jul 13 19:05:50.638188 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:50.638097 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-574m7" Jul 13 19:05:50.813302 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:50.813265 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-574m7"] Jul 13 19:05:51.234346 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:51.234307 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-574m7" event={"ID":"1135c060-7bcd-4879-8212-4527f428e844","Type":"ContainerStarted","Data":"6a8b93761cf84d2bc50b9c669335de787f1f5d6ba2aa3304f22897463dc1d59a"} Jul 13 19:05:53.245170 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:53.245076 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-574m7" event={"ID":"1135c060-7bcd-4879-8212-4527f428e844","Type":"ContainerStarted","Data":"8d03c59a53293a0898d28f6935fff27b06ec694764d2b457e64573a1cb92e30d"} Jul 13 19:05:53.245557 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:53.245270 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-574m7" Jul 13 19:05:54.668504 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:54.668466 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:54.770752 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:54.770719 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:59.683620 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:59.683490 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:05:59.959735 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:05:59.959649 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:00.084713 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:00.084679 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:00.209864 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:00.209748 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:03.852391 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:03.852360 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:04.087546 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:04.087515 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:04.251758 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:04.251677 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-574m7" Jul 13 19:06:04.269334 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:04.269288 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-574m7" podStartSLOduration=12.174340074 podStartE2EDuration="14.269272417s" totalImagesPullingTime="2.094932343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:05:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:05:53.272443366 +0000 UTC m=+438.748897555" watchObservedRunningTime="2026-07-13 19:06:04.269272417 +0000 UTC m=+449.745726591" Jul 13 19:06:07.450672 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:07.450634 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:07.550677 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:07.550639 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:19.075611 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:19.075577 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:19.206156 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:19.206115 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:30.581681 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:30.581649 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:35.448644 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:35.448611 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:35.523203 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:35.523168 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:38.436838 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:38.436801 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:38.612719 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:38.612679 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:38.717966 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:38.717855 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:39.026669 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:39.026589 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:06:39.030160 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:39.030142 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-k82x4" Jul 13 19:06:39.034418 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:39.034393 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lsvdw\"" Jul 13 19:06:39.034562 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:39.034467 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 13 19:06:39.048879 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:39.048853 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:06:39.071011 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:39.070979 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:06:39.141508 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:39.141473 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e6559d43-6f70-440d-81fb-dbef1542cad1-config-file\") pod \"limitador-limitador-54d6d988d5-k82x4\" (UID: \"e6559d43-6f70-440d-81fb-dbef1542cad1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-k82x4" Jul 13 19:06:39.141683 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:39.141514 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzrql\" (UniqueName: \"kubernetes.io/projected/e6559d43-6f70-440d-81fb-dbef1542cad1-kube-api-access-tzrql\") pod \"limitador-limitador-54d6d988d5-k82x4\" (UID: \"e6559d43-6f70-440d-81fb-dbef1542cad1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-k82x4" Jul 13 19:06:39.243141 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:39.243106 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e6559d43-6f70-440d-81fb-dbef1542cad1-config-file\") pod \"limitador-limitador-54d6d988d5-k82x4\" (UID: \"e6559d43-6f70-440d-81fb-dbef1542cad1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-k82x4" Jul 13 19:06:39.243297 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:39.243147 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzrql\" (UniqueName: \"kubernetes.io/projected/e6559d43-6f70-440d-81fb-dbef1542cad1-kube-api-access-tzrql\") pod \"limitador-limitador-54d6d988d5-k82x4\" (UID: \"e6559d43-6f70-440d-81fb-dbef1542cad1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-k82x4" Jul 13 19:06:39.243771 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:39.243753 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e6559d43-6f70-440d-81fb-dbef1542cad1-config-file\") pod \"limitador-limitador-54d6d988d5-k82x4\" (UID: \"e6559d43-6f70-440d-81fb-dbef1542cad1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-k82x4" Jul 13 19:06:39.255667 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:39.255635 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzrql\" (UniqueName: \"kubernetes.io/projected/e6559d43-6f70-440d-81fb-dbef1542cad1-kube-api-access-tzrql\") pod \"limitador-limitador-54d6d988d5-k82x4\" (UID: \"e6559d43-6f70-440d-81fb-dbef1542cad1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-k82x4" Jul 13 19:06:39.341212 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:39.341179 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-k82x4" Jul 13 19:06:39.482634 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:39.482607 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:06:39.484986 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:06:39.484956 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6559d43_6f70_440d_81fb_dbef1542cad1.slice/crio-0e22f25e0085cb95858d01c0a98085baae9baebc379ca26c6440f3fd7b12fdd3 WatchSource:0}: Error finding container 0e22f25e0085cb95858d01c0a98085baae9baebc379ca26c6440f3fd7b12fdd3: Status 404 returned error can't find the container with id 0e22f25e0085cb95858d01c0a98085baae9baebc379ca26c6440f3fd7b12fdd3 Jul 13 19:06:39.486291 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:39.486242 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:39.595577 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:39.595494 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:40.472260 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:40.472185 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-k82x4" event={"ID":"e6559d43-6f70-440d-81fb-dbef1542cad1","Type":"ContainerStarted","Data":"0e22f25e0085cb95858d01c0a98085baae9baebc379ca26c6440f3fd7b12fdd3"} Jul 13 19:06:42.484921 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:42.484880 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-k82x4" event={"ID":"e6559d43-6f70-440d-81fb-dbef1542cad1","Type":"ContainerStarted","Data":"172230b80e54e39656b3d5c05f72221b741ddc7bb597bb3b7ece3d84fd306c0f"} Jul 13 19:06:42.485392 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:42.484991 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-k82x4" Jul 13 19:06:51.607997 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:51.607965 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:51.813841 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:51.813779 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:53.490251 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:53.490217 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-k82x4" Jul 13 19:06:53.518363 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:53.518311 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-k82x4" podStartSLOduration=13.012387353 podStartE2EDuration="15.518294775s" totalImagesPullingTime="2.505907422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:06:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:06:42.528099187 +0000 UTC m=+488.004553379" watchObservedRunningTime="2026-07-13 19:06:53.518294775 +0000 UTC m=+498.994748948" Jul 13 19:06:55.331970 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:55.331939 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:55.438113 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:55.438080 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:06:55.865102 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:06:55.865074 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:08.316509 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:07:08.316473 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:08.531745 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:07:08.531712 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:16.281841 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:07:16.281779 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:16.428882 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:07:16.428846 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:16.894157 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:07:16.894125 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:19.721030 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:07:19.720990 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:19.836991 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:07:19.836956 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:49.262455 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:07:49.262383 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:49.356251 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:07:49.356223 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:49.750028 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:07:49.749996 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:50.112822 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:07:50.112771 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:50.422436 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:07:50.422358 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:50.565515 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:07:50.565481 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:07:51.636016 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:07:51.635973 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:59.529625 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:07:59.529552 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:07:59.626423 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:07:59.626389 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:11.227538 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:08:11.227481 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:11.327667 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:08:11.327614 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:16.989266 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:08:16.989235 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:23.076901 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:08:23.076859 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:23.644040 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:08:23.644005 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:08:31.175471 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:08:31.175430 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:32.049726 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:08:32.049689 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:08:34.939584 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:08:34.939556 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 19:08:34.945034 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:08:34.945013 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 19:08:45.398568 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:08:45.398534 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:46.553075 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:08:46.553038 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:08:56.585538 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:08:56.585495 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:08:58.453982 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:08:58.453948 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:09:05.828493 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:05.828458 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:19.400997 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:19.400962 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:21.447604 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:21.447571 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:09:27.248264 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.248229 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns"] Jul 13 19:09:27.251586 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.251568 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.254851 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.254825 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 13 19:09:27.254851 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.254841 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 13 19:09:27.255064 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.254835 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-j6mtd\"" Jul 13 19:09:27.255064 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.254856 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 13 19:09:27.265066 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.265042 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns"] Jul 13 19:09:27.276405 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.276371 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:09:27.297596 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.297569 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qfth\" (UniqueName: \"kubernetes.io/projected/85de423e-4a1e-4cc4-a91c-c908f815e28e-kube-api-access-7qfth\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns\" (UID: \"85de423e-4a1e-4cc4-a91c-c908f815e28e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.297716 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.297623 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85de423e-4a1e-4cc4-a91c-c908f815e28e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns\" (UID: \"85de423e-4a1e-4cc4-a91c-c908f815e28e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.297716 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.297699 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85de423e-4a1e-4cc4-a91c-c908f815e28e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns\" (UID: \"85de423e-4a1e-4cc4-a91c-c908f815e28e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.297814 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.297762 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85de423e-4a1e-4cc4-a91c-c908f815e28e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns\" (UID: \"85de423e-4a1e-4cc4-a91c-c908f815e28e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.297856 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.297815 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85de423e-4a1e-4cc4-a91c-c908f815e28e-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns\" (UID: \"85de423e-4a1e-4cc4-a91c-c908f815e28e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.297856 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.297838 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85de423e-4a1e-4cc4-a91c-c908f815e28e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns\" (UID: \"85de423e-4a1e-4cc4-a91c-c908f815e28e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.398865 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.398813 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85de423e-4a1e-4cc4-a91c-c908f815e28e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns\" (UID: \"85de423e-4a1e-4cc4-a91c-c908f815e28e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.398865 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.398880 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85de423e-4a1e-4cc4-a91c-c908f815e28e-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns\" (UID: \"85de423e-4a1e-4cc4-a91c-c908f815e28e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.399113 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.398907 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85de423e-4a1e-4cc4-a91c-c908f815e28e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns\" (UID: \"85de423e-4a1e-4cc4-a91c-c908f815e28e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.399113 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.398930 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qfth\" (UniqueName: \"kubernetes.io/projected/85de423e-4a1e-4cc4-a91c-c908f815e28e-kube-api-access-7qfth\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns\" (UID: \"85de423e-4a1e-4cc4-a91c-c908f815e28e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.399113 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.399003 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85de423e-4a1e-4cc4-a91c-c908f815e28e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns\" (UID: \"85de423e-4a1e-4cc4-a91c-c908f815e28e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.399113 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.399064 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85de423e-4a1e-4cc4-a91c-c908f815e28e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns\" (UID: \"85de423e-4a1e-4cc4-a91c-c908f815e28e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.399317 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.399220 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85de423e-4a1e-4cc4-a91c-c908f815e28e-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns\" (UID: \"85de423e-4a1e-4cc4-a91c-c908f815e28e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.399317 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.399250 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85de423e-4a1e-4cc4-a91c-c908f815e28e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns\" (UID: \"85de423e-4a1e-4cc4-a91c-c908f815e28e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.399429 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.399406 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85de423e-4a1e-4cc4-a91c-c908f815e28e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns\" (UID: \"85de423e-4a1e-4cc4-a91c-c908f815e28e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.401178 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.401157 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85de423e-4a1e-4cc4-a91c-c908f815e28e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns\" (UID: \"85de423e-4a1e-4cc4-a91c-c908f815e28e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.401421 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.401407 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85de423e-4a1e-4cc4-a91c-c908f815e28e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns\" (UID: \"85de423e-4a1e-4cc4-a91c-c908f815e28e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.407275 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.407239 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qfth\" (UniqueName: \"kubernetes.io/projected/85de423e-4a1e-4cc4-a91c-c908f815e28e-kube-api-access-7qfth\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns\" (UID: \"85de423e-4a1e-4cc4-a91c-c908f815e28e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.561098 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.561068 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:27.696996 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.696968 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns"] Jul 13 19:09:27.698516 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:09:27.698481 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85de423e_4a1e_4cc4_a91c_c908f815e28e.slice/crio-78d178791f54a245227d3e834dfe503afa01dde7c8d1052d77650eaebcbaf220 WatchSource:0}: Error finding container 78d178791f54a245227d3e834dfe503afa01dde7c8d1052d77650eaebcbaf220: Status 404 returned error can't find the container with id 78d178791f54a245227d3e834dfe503afa01dde7c8d1052d77650eaebcbaf220 Jul 13 19:09:27.699854 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:27.699833 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 19:09:28.269492 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:28.269455 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" event={"ID":"85de423e-4a1e-4cc4-a91c-c908f815e28e","Type":"ContainerStarted","Data":"78d178791f54a245227d3e834dfe503afa01dde7c8d1052d77650eaebcbaf220"} Jul 13 19:09:29.276033 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:29.275996 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" event={"ID":"85de423e-4a1e-4cc4-a91c-c908f815e28e","Type":"ContainerStarted","Data":"a48046662cb4d429da5d1c4bc23305f2335d3212d09485c31c9f10354c733657"} Jul 13 19:09:29.299603 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:29.299552 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" podStartSLOduration=1.261745116 podStartE2EDuration="2.299535679s" totalImagesPullingTime="1.037790563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:09:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:09:29.297739674 +0000 UTC m=+654.774193848" watchObservedRunningTime="2026-07-13 19:09:29.299535679 +0000 UTC m=+654.775989852" Jul 13 19:09:30.063568 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:30.063538 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:09:37.561238 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:37.561201 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:37.561723 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:37.561365 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:37.573532 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:37.573506 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:09:38.330745 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:09:38.330711 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns" Jul 13 19:10:21.531541 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:10:21.531507 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:21.636906 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:10:21.636875 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:22.664325 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:10:22.664292 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:22.763844 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:10:22.763809 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:10:51.649127 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:10:51.649051 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:11:03.261580 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:11:03.261546 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:11:13.945697 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:11:13.945653 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:11:17.593554 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:11:17.593520 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:23.162093 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:11:23.162052 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:11:33.256663 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:11:33.256623 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:11:34.656605 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:11:34.656570 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:11:36.424091 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:11:36.424054 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:00.164900 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:12:00.164818 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:12:01.646847 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:12:01.646800 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:01.744000 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:12:01.743960 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:15.854672 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:12:15.854635 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:12:20.851618 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:12:20.851582 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:20.949154 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:12:20.949121 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:21.058981 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:12:21.058944 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:26.922866 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:12:26.922829 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:27.034988 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:12:27.034947 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:27.123910 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:12:27.123873 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:27.302906 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:12:27.302877 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:32.954063 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:12:32.954024 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:33.056544 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:12:33.056507 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:12:49.859170 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:12:49.859073 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:13:00.346173 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:13:00.346131 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:13:10.956099 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:13:10.956058 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:13:18.554890 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:13:18.554858 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:13:30.559108 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:13:30.559027 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:13:34.973427 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:13:34.973387 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 19:13:34.976053 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:13:34.976027 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 19:13:40.854222 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:13:40.854186 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:13:53.952088 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:13:53.952049 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:14:02.241355 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:14:02.241315 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:14:25.057206 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:14:25.057173 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:14:36.654028 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:14:36.653989 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:14:48.105623 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:14:48.105586 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:14:55.236673 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:14:55.236630 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:15:00.044823 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:15:00.044710 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:15:12.345139 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:15:12.345103 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:15:28.959140 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:15:28.959103 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:15:38.549707 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:15:38.549670 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:15:49.148554 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:15:49.148524 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:15:53.416538 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:15:53.416502 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:15:53.622065 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:15:53.622022 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:15:59.446871 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:15:59.446836 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:16:09.745972 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:16:09.745933 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:17:55.260403 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:17:55.260369 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:18:06.454687 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:18:06.454645 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:18:27.573644 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:18:27.573608 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:18:35.006831 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:18:35.006802 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 19:18:35.006831 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:18:35.006815 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 19:18:38.295148 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:18:38.295110 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:18:48.661556 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:18:48.661521 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:18:57.668251 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:18:57.668212 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:19:07.965196 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:19:07.965160 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:19:18.261490 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:19:18.261453 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:19:28.065097 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:19:28.065014 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:19:42.368721 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:19:42.368686 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:19:59.855854 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:19:59.855813 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:20:09.762466 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:20:09.762424 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:20:20.564291 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:20:20.564256 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:20:33.945535 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:20:33.945498 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:20:51.150155 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:20:51.150116 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:21:01.551883 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:21:01.551796 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:21:11.147153 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:21:11.147111 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:21:11.654905 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:21:11.654868 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:21:12.457549 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:21:12.457502 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:21:21.081697 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:21:21.081661 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:21:30.557518 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:21:30.557484 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:21:42.287167 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:21:42.287131 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:21:42.655911 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:21:42.655872 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:22:00.450055 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:22:00.450011 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:22:00.630047 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:22:00.629999 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:22:18.952239 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:22:18.952205 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:22:27.365414 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:22:27.365324 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:22:37.976125 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:22:37.976086 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:22:56.859589 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:22:56.859550 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:23:08.949733 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:23:08.949694 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:23:21.150526 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:23:21.150484 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:23:26.426274 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:23:26.426232 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:23:30.954748 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:23:30.954714 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:23:35.041029 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:23:35.041004 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 19:23:35.041855 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:23:35.041835 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 19:23:36.337737 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:23:36.337705 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:23:41.163841 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:23:41.163805 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:23:44.621797 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:23:44.621742 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:23:46.909195 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:23:46.909164 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:23:49.748365 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:23:49.748313 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:23:52.861388 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:23:52.861342 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:23:53.011629 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:23:53.011593 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:23:53.346345 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:23:53.346315 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:23:54.678377 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:23:54.678344 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:24:00.164385 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:00.164345 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:00.959757 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:00.959667 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:01.087017 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:01.086988 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:01.834273 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:01.834238 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:01.989360 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:01.989325 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:02.119058 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:02.118963 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:02.273257 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:02.273226 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:02.981474 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:02.981443 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:03.154446 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:03.154405 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:04.791310 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:04.791280 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:06.058057 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:06.058024 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:24:17.626047 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:17.626007 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:22.004480 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:22.004446 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:23.473106 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:23.473070 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:24:28.005101 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:28.005064 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:28.109415 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:28.109375 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:32.199575 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:32.199534 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:32.567239 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:32.567204 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:32.669743 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:32.669701 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:32.819805 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:32.819702 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:34.198956 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:34.198920 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:34.371307 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:34.371272 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:34.583129 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:34.583094 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:34.895009 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:34.894928 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:35.002001 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:35.001967 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:38.902695 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:38.902660 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:38.974086 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:38.974033 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:40.250161 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:40.250130 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:24:41.906219 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:41.906186 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:24:44.318989 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:24:44.318958 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:25:38.711620 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:25:38.711514 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:25:38.824765 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:25:38.824730 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:25:39.143626 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:25:39.143592 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:25:41.853211 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:25:41.853178 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:25:45.069434 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:25:45.069394 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:25:45.174374 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:25:45.174318 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:25:45.298067 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:25:45.298032 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:25:45.388529 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:25:45.388434 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:25:46.457392 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:25:46.457350 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:25:46.549343 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:25:46.549306 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:25:46.724158 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:25:46.724073 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:25:46.992829 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:25:46.992713 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:25:47.468895 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:25:47.468860 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:25:47.575230 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:25:47.575185 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:25:48.896990 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:25:48.896953 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:25:50.246828 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:25:50.246776 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:26:37.454466 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:26:37.454432 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:26:39.221211 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:26:39.221175 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:26:52.314374 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:26:52.314341 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:26:55.257091 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:26:55.257050 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:26:56.747333 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:26:56.747299 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:26:58.260933 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:26:58.260896 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:26:58.399853 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:26:58.399814 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:27:02.862428 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:27:02.862389 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:27:03.027940 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:27:03.027901 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:27:07.833061 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:27:07.833022 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:27:08.100863 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:27:08.100766 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:27:09.274552 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:27:09.274517 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:27:09.374774 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:27:09.374737 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:27:09.912464 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:27:09.912434 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:27:10.024192 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:27:10.024158 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:27:10.568046 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:27:10.568019 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:27:11.156873 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:27:11.156841 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:27:12.359336 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:27:12.359302 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:27:14.448230 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:27:14.448189 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:27:14.547716 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:27:14.547682 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:27:17.517978 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:27:17.517940 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:28:04.660266 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:04.660203 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:28:14.163194 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:14.163157 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:28:16.952138 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:16.952104 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:28:18.781119 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:18.781076 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:28:21.454616 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:21.454577 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:28:24.793352 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:24.793315 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:28:24.950736 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:24.950703 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:28:29.014402 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:29.014324 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:28:29.624445 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:29.624411 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:28:29.732248 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:29.732209 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:28:29.829147 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:29.829114 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:28:31.264985 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:31.264941 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:28:35.073604 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:35.073573 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 19:28:35.076019 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:35.075999 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 19:28:35.377190 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:35.377157 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:28:37.573275 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:37.573245 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:28:37.677705 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:37.677671 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:28:37.793567 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:37.793531 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:28:38.421326 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:38.421292 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:28:38.521316 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:28:38.521281 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:04.416887 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:04.416840 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:10.379099 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:10.379064 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:10.486402 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:10.486366 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:10.585839 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:10.585809 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:26.509754 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:26.509718 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:32.365401 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:32.365361 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:32.437713 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:32.437680 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:32.543698 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:32.543665 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:43.121841 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:43.121807 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:44.898327 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:44.898293 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:50.003171 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:50.003123 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:50.108663 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:50.108630 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:50.204911 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:50.204873 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:50.304146 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:50.304112 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:51.087631 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:51.087592 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:51.179251 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:51.179205 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:51.397725 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:51.397640 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:52.969574 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:52.969544 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:53.749070 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:53.749032 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:29:59.328169 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:59.328092 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:29:59.416391 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:29:59.416359 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:00.886583 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:00.886538 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:03.658914 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:03.658881 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:30:06.117241 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:06.117207 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:06.213830 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:06.213778 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:06.386424 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:06.386341 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:06.857769 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:06.857734 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:07.103084 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:07.103050 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:10.059572 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:10.059535 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:10.213326 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:10.213291 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:10.421512 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:10.421423 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:10.711840 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:10.711690 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:11.086177 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:11.086147 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:11.187396 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:11.187354 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:14.577501 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:14.577461 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:15.234972 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:15.234941 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:15.332571 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:15.332542 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:15.659322 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:15.659283 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:15.866422 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:15.866388 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:30:21.468894 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:21.468857 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:21.547243 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:21.547201 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:32.421015 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:32.420959 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:35.744956 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:35.744915 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:30:36.953968 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:36.953926 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:43.134456 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:43.134422 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:44.547402 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:44.547367 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:47.666148 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:47.666112 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:47.895652 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:47.895616 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:48.009204 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:48.009116 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:49.463634 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:49.463597 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:49.560976 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:49.560944 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:54.131960 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:54.131916 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:54.239965 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:54.239935 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:55.576078 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:55.576044 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:30:55.678551 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:30:55.678520 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:31:00.691888 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:00.691853 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:31:01.922545 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:01.922511 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:31:07.553518 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:07.553480 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:31:07.944476 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:07.944394 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:31:08.156766 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:08.156732 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:31:11.455755 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:11.455711 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:31:11.563100 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:11.563070 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:31:11.664424 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:11.664392 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:31:12.065761 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:12.065721 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:31:12.166075 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:12.166043 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:31:12.268436 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:12.268400 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:31:14.047178 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:14.047136 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:31:15.819981 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:15.819941 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:31:26.954894 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:26.954806 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7"] Jul 13 19:31:26.958758 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:26.958738 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:26.961215 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:26.961190 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-97bb288c\"/\"kube-root-ca.crt\"" Jul 13 19:31:26.962158 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:26.962137 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-97bb288c\"/\"openshift-service-ca.crt\"" Jul 13 19:31:26.962283 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:26.962163 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-97bb288c\"/\"e2e-lifecycle-model-97bb288c-kserve-self-signed-certs\"" Jul 13 19:31:26.962283 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:26.962139 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-97bb288c\"/\"default-dockercfg-mkrbp\"" Jul 13 19:31:26.969632 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:26.969606 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7"] Jul 13 19:31:26.987147 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:26.987120 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:31:27.032579 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.032546 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pvvd\" (UniqueName: \"kubernetes.io/projected/cc45ad67-1375-4ad7-89f7-50172481418c-kube-api-access-5pvvd\") pod \"e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.032737 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.032587 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-model-cache\") pod \"e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.032737 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.032610 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc45ad67-1375-4ad7-89f7-50172481418c-tls-certs\") pod \"e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.032737 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.032692 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-tmp-dir\") pod \"e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.032737 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.032712 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-dshm\") pod \"e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.032916 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.032769 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-home\") pod \"e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.134292 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.134262 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-home\") pod \"e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.134492 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.134323 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pvvd\" (UniqueName: \"kubernetes.io/projected/cc45ad67-1375-4ad7-89f7-50172481418c-kube-api-access-5pvvd\") pod \"e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.134492 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.134357 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-model-cache\") pod \"e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.134492 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.134385 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc45ad67-1375-4ad7-89f7-50172481418c-tls-certs\") pod \"e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.134492 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.134440 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-tmp-dir\") pod \"e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.134492 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.134463 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-dshm\") pod \"e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.134829 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.134770 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-home\") pod \"e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.134889 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.134862 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-tmp-dir\") pod \"e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.135194 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.135170 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-model-cache\") pod \"e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.137394 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.137363 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-dshm\") pod \"e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.137713 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.137691 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc45ad67-1375-4ad7-89f7-50172481418c-tls-certs\") pod \"e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.142759 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.142734 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pvvd\" (UniqueName: \"kubernetes.io/projected/cc45ad67-1375-4ad7-89f7-50172481418c-kube-api-access-5pvvd\") pod \"e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.268670 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.268582 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:27.400370 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.400344 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7"] Jul 13 19:31:27.402858 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:31:27.402825 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc45ad67_1375_4ad7_89f7_50172481418c.slice/crio-9598bdf234719cb8606af404051c93b3661971ccd53865374a048e7e00b83c9b WatchSource:0}: Error finding container 9598bdf234719cb8606af404051c93b3661971ccd53865374a048e7e00b83c9b: Status 404 returned error can't find the container with id 9598bdf234719cb8606af404051c93b3661971ccd53865374a048e7e00b83c9b Jul 13 19:31:27.696536 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.696500 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" event={"ID":"cc45ad67-1375-4ad7-89f7-50172481418c","Type":"ContainerStarted","Data":"1286c124a318684ba1cd510bf4c34da4ca01d2a43051a3f90108e0ec81b87ec1"} Jul 13 19:31:27.696723 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.696548 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" event={"ID":"cc45ad67-1375-4ad7-89f7-50172481418c","Type":"ContainerStarted","Data":"9598bdf234719cb8606af404051c93b3661971ccd53865374a048e7e00b83c9b"} Jul 13 19:31:27.716118 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:27.716065 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" podStartSLOduration=1.7160482510000001 podStartE2EDuration="1.716048251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:31:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:31:27.714334206 +0000 UTC m=+1973.190788381" watchObservedRunningTime="2026-07-13 19:31:27.716048251 +0000 UTC m=+1973.192502425" Jul 13 19:31:33.144466 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:33.144427 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:31:37.268922 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:37.268874 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:37.269302 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:37.268936 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:37.281179 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:37.281150 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:37.754812 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:37.754770 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:31:46.150108 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:46.150066 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:31:58.991434 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:31:58.991399 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:32:02.997182 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:02.997130 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:32:03.747938 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:03.747898 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:32:04.301168 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:04.301129 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:32:09.304133 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:09.304097 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7"] Jul 13 19:32:09.352170 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:09.352137 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:32:24.325111 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.325060 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" podUID="cc45ad67-1375-4ad7-89f7-50172481418c" containerName="main" containerID="cri-o://1286c124a318684ba1cd510bf4c34da4ca01d2a43051a3f90108e0ec81b87ec1" gracePeriod=45 Jul 13 19:32:24.580302 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.580239 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:32:24.673173 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.673136 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-model-cache\") pod \"cc45ad67-1375-4ad7-89f7-50172481418c\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " Jul 13 19:32:24.673358 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.673215 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-home\") pod \"cc45ad67-1375-4ad7-89f7-50172481418c\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " Jul 13 19:32:24.673358 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.673246 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc45ad67-1375-4ad7-89f7-50172481418c-tls-certs\") pod \"cc45ad67-1375-4ad7-89f7-50172481418c\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " Jul 13 19:32:24.673358 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.673283 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-tmp-dir\") pod \"cc45ad67-1375-4ad7-89f7-50172481418c\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " Jul 13 19:32:24.673358 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.673303 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pvvd\" (UniqueName: \"kubernetes.io/projected/cc45ad67-1375-4ad7-89f7-50172481418c-kube-api-access-5pvvd\") pod \"cc45ad67-1375-4ad7-89f7-50172481418c\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " Jul 13 19:32:24.673358 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.673322 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-dshm\") pod \"cc45ad67-1375-4ad7-89f7-50172481418c\" (UID: \"cc45ad67-1375-4ad7-89f7-50172481418c\") " Jul 13 19:32:24.673629 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.673443 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-model-cache" (OuterVolumeSpecName: "model-cache") pod "cc45ad67-1375-4ad7-89f7-50172481418c" (UID: "cc45ad67-1375-4ad7-89f7-50172481418c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:32:24.673629 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.673521 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cc45ad67-1375-4ad7-89f7-50172481418c" (UID: "cc45ad67-1375-4ad7-89f7-50172481418c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:32:24.673629 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.673536 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-home" (OuterVolumeSpecName: "home") pod "cc45ad67-1375-4ad7-89f7-50172481418c" (UID: "cc45ad67-1375-4ad7-89f7-50172481418c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:32:24.673767 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.673645 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:32:24.673767 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.673663 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:32:24.673767 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.673678 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:32:24.675513 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.675484 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-dshm" (OuterVolumeSpecName: "dshm") pod "cc45ad67-1375-4ad7-89f7-50172481418c" (UID: "cc45ad67-1375-4ad7-89f7-50172481418c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:32:24.675636 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.675549 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc45ad67-1375-4ad7-89f7-50172481418c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cc45ad67-1375-4ad7-89f7-50172481418c" (UID: "cc45ad67-1375-4ad7-89f7-50172481418c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:32:24.675636 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.675564 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc45ad67-1375-4ad7-89f7-50172481418c-kube-api-access-5pvvd" (OuterVolumeSpecName: "kube-api-access-5pvvd") pod "cc45ad67-1375-4ad7-89f7-50172481418c" (UID: "cc45ad67-1375-4ad7-89f7-50172481418c"). InnerVolumeSpecName "kube-api-access-5pvvd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:32:24.774118 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.774092 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc45ad67-1375-4ad7-89f7-50172481418c-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:32:24.774118 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.774116 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pvvd\" (UniqueName: \"kubernetes.io/projected/cc45ad67-1375-4ad7-89f7-50172481418c-kube-api-access-5pvvd\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:32:24.774118 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.774125 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc45ad67-1375-4ad7-89f7-50172481418c-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:32:24.972732 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.972628 2574 generic.go:356] "Generic (PLEG): container finished" podID="cc45ad67-1375-4ad7-89f7-50172481418c" containerID="1286c124a318684ba1cd510bf4c34da4ca01d2a43051a3f90108e0ec81b87ec1" exitCode=0 Jul 13 19:32:24.972732 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.972704 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" Jul 13 19:32:24.972732 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.972717 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" event={"ID":"cc45ad67-1375-4ad7-89f7-50172481418c","Type":"ContainerDied","Data":"1286c124a318684ba1cd510bf4c34da4ca01d2a43051a3f90108e0ec81b87ec1"} Jul 13 19:32:24.973043 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.972761 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7" event={"ID":"cc45ad67-1375-4ad7-89f7-50172481418c","Type":"ContainerDied","Data":"9598bdf234719cb8606af404051c93b3661971ccd53865374a048e7e00b83c9b"} Jul 13 19:32:24.973043 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.972797 2574 scope.go:117] "RemoveContainer" containerID="1286c124a318684ba1cd510bf4c34da4ca01d2a43051a3f90108e0ec81b87ec1" Jul 13 19:32:24.982331 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.982312 2574 scope.go:117] "RemoveContainer" containerID="1286c124a318684ba1cd510bf4c34da4ca01d2a43051a3f90108e0ec81b87ec1" Jul 13 19:32:24.982588 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:32:24.982570 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1286c124a318684ba1cd510bf4c34da4ca01d2a43051a3f90108e0ec81b87ec1\": container with ID starting with 1286c124a318684ba1cd510bf4c34da4ca01d2a43051a3f90108e0ec81b87ec1 not found: ID does not exist" containerID="1286c124a318684ba1cd510bf4c34da4ca01d2a43051a3f90108e0ec81b87ec1" Jul 13 19:32:24.982639 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.982596 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1286c124a318684ba1cd510bf4c34da4ca01d2a43051a3f90108e0ec81b87ec1"} err="failed to get container status \"1286c124a318684ba1cd510bf4c34da4ca01d2a43051a3f90108e0ec81b87ec1\": rpc error: code = NotFound desc = could not find container \"1286c124a318684ba1cd510bf4c34da4ca01d2a43051a3f90108e0ec81b87ec1\": container with ID starting with 1286c124a318684ba1cd510bf4c34da4ca01d2a43051a3f90108e0ec81b87ec1 not found: ID does not exist" Jul 13 19:32:24.996865 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.996835 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7"] Jul 13 19:32:24.998749 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:24.998728 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-97bb288c/e2e-lifecycle-model-97bb288c-kserve-77d9776fff-rqvz7"] Jul 13 19:32:25.056577 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:25.056544 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc45ad67-1375-4ad7-89f7-50172481418c" path="/var/lib/kubelet/pods/cc45ad67-1375-4ad7-89f7-50172481418c/volumes" Jul 13 19:32:46.366440 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:46.366400 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:32:46.484417 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:46.484382 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:32:52.266158 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:52.266116 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:32:52.366332 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:52.366286 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:32:59.128867 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:59.128753 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:32:59.222654 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:32:59.222610 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:00.350140 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:00.350084 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:00.851802 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:00.851752 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:33:06.411436 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:06.411396 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:06.514408 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:06.514357 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:07.093770 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:07.093727 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:13.271798 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:13.271734 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:13.391487 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:13.391447 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:13.936153 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:13.936115 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:15.292188 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:15.292147 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:33:24.098288 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:24.098254 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:24.467354 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:24.467274 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:35.105655 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:35.105627 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 19:33:35.109477 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:35.109456 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 19:33:45.468341 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:45.468304 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:51.324237 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:51.324187 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:51.428431 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:51.428399 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:51.679982 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:51.679866 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:52.028361 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:52.028269 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:52.298309 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:52.298271 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:55.277776 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:55.277743 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:55.395003 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:55.394974 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:55.480407 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:55.480376 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:55.608472 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:55.608438 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:55.955155 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:55.955055 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:56.054975 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:56.054941 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:33:59.648960 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:59.648928 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:33:59.781298 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:33:59.781261 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:00.147696 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:00.147665 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:00.672115 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:00.671895 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:01.150267 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:01.150233 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:03.455840 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:03.455807 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:03.666311 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:03.666280 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:03.798196 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:03.798170 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:04.465623 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:04.465516 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:04.620162 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:04.620127 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:07.904867 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:07.904834 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:12.757769 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:12.757730 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:12.815362 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:12.815323 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:14.949851 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:14.949816 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:34:16.906008 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:16.905971 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:17.007232 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.007202 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:17.530514 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.530482 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426"] Jul 13 19:34:17.530917 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.530903 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc45ad67-1375-4ad7-89f7-50172481418c" containerName="main" Jul 13 19:34:17.530917 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.530918 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc45ad67-1375-4ad7-89f7-50172481418c" containerName="main" Jul 13 19:34:17.531029 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.530972 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc45ad67-1375-4ad7-89f7-50172481418c" containerName="main" Jul 13 19:34:17.534182 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.534166 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.537347 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.537327 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d0bb2f\"/\"kube-root-ca.crt\"" Jul 13 19:34:17.537542 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.537327 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d0bb2f\"/\"openshift-service-ca.crt\"" Jul 13 19:34:17.537661 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.537350 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d0bb2f\"/\"default-dockercfg-hfk84\"" Jul 13 19:34:17.537661 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.537347 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d0bb2f\"/\"test-model-d0bb2f-kserve-self-signed-certs\"" Jul 13 19:34:17.545898 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.545875 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426"] Jul 13 19:34:17.638691 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.638649 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:17.639775 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.639751 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-tmp-dir\") pod \"test-model-d0bb2f-kserve-6bdccddd77-df426\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.639914 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.639807 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glz9z\" (UniqueName: \"kubernetes.io/projected/b16c2793-2168-4ef7-a249-438d1fd2be50-kube-api-access-glz9z\") pod \"test-model-d0bb2f-kserve-6bdccddd77-df426\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.639914 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.639868 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-model-cache\") pod \"test-model-d0bb2f-kserve-6bdccddd77-df426\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.640010 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.639923 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-dshm\") pod \"test-model-d0bb2f-kserve-6bdccddd77-df426\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.640010 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.639978 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-home\") pod \"test-model-d0bb2f-kserve-6bdccddd77-df426\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.640113 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.640078 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b16c2793-2168-4ef7-a249-438d1fd2be50-tls-certs\") pod \"test-model-d0bb2f-kserve-6bdccddd77-df426\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.741028 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.740986 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-model-cache\") pod \"test-model-d0bb2f-kserve-6bdccddd77-df426\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.741231 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.741036 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-dshm\") pod \"test-model-d0bb2f-kserve-6bdccddd77-df426\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.741231 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.741096 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-home\") pod \"test-model-d0bb2f-kserve-6bdccddd77-df426\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.741231 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.741183 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b16c2793-2168-4ef7-a249-438d1fd2be50-tls-certs\") pod \"test-model-d0bb2f-kserve-6bdccddd77-df426\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.741231 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.741216 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-tmp-dir\") pod \"test-model-d0bb2f-kserve-6bdccddd77-df426\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.741448 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.741243 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glz9z\" (UniqueName: \"kubernetes.io/projected/b16c2793-2168-4ef7-a249-438d1fd2be50-kube-api-access-glz9z\") pod \"test-model-d0bb2f-kserve-6bdccddd77-df426\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.741448 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.741414 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-model-cache\") pod \"test-model-d0bb2f-kserve-6bdccddd77-df426\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.741547 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.741529 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-tmp-dir\") pod \"test-model-d0bb2f-kserve-6bdccddd77-df426\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.741597 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.741552 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-home\") pod \"test-model-d0bb2f-kserve-6bdccddd77-df426\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.743312 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.743289 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-dshm\") pod \"test-model-d0bb2f-kserve-6bdccddd77-df426\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.743649 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.743630 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b16c2793-2168-4ef7-a249-438d1fd2be50-tls-certs\") pod \"test-model-d0bb2f-kserve-6bdccddd77-df426\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.749605 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.749586 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glz9z\" (UniqueName: \"kubernetes.io/projected/b16c2793-2168-4ef7-a249-438d1fd2be50-kube-api-access-glz9z\") pod \"test-model-d0bb2f-kserve-6bdccddd77-df426\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.844129 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.844091 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:17.981738 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:17.981713 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426"] Jul 13 19:34:17.983704 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:34:17.983677 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb16c2793_2168_4ef7_a249_438d1fd2be50.slice/crio-d07382194e09c2a3dabe81d52d7663d2df80bf52e9c4fed5ef8b40e11eef2359 WatchSource:0}: Error finding container d07382194e09c2a3dabe81d52d7663d2df80bf52e9c4fed5ef8b40e11eef2359: Status 404 returned error can't find the container with id d07382194e09c2a3dabe81d52d7663d2df80bf52e9c4fed5ef8b40e11eef2359 Jul 13 19:34:18.534898 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:18.534861 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" event={"ID":"b16c2793-2168-4ef7-a249-438d1fd2be50","Type":"ContainerStarted","Data":"e10e46356b3cf2d3e6a9a426fe959f426fd61a87b617073b57643a8e3b3aa623"} Jul 13 19:34:18.535082 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:18.534906 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" event={"ID":"b16c2793-2168-4ef7-a249-438d1fd2be50","Type":"ContainerStarted","Data":"d07382194e09c2a3dabe81d52d7663d2df80bf52e9c4fed5ef8b40e11eef2359"} Jul 13 19:34:18.553720 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:18.553665 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" podStartSLOduration=1.5536501870000001 podStartE2EDuration="1.553650187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:34:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:34:18.552567434 +0000 UTC m=+2144.029021609" watchObservedRunningTime="2026-07-13 19:34:18.553650187 +0000 UTC m=+2144.030104361" Jul 13 19:34:25.247381 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:25.247345 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:34:25.384082 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:25.384048 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:25.484686 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:25.484648 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:27.844914 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:27.844836 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:27.844914 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:27.844885 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:27.857252 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:27.857227 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:28.597672 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:28.597642 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:34:36.244330 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.244296 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6"] Jul 13 19:34:36.248056 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.248034 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.250174 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.250154 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0a1747\"/\"openshift-service-ca.crt\"" Jul 13 19:34:36.250174 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.250165 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0a1747\"/\"test-model-0a1747-kserve-self-signed-certs\"" Jul 13 19:34:36.250953 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.250933 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0a1747\"/\"kube-root-ca.crt\"" Jul 13 19:34:36.251048 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.250996 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0a1747\"/\"default-dockercfg-j5k2w\"" Jul 13 19:34:36.257552 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.257527 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6"] Jul 13 19:34:36.282432 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.282390 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:36.297448 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.297413 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-model-cache\") pod \"test-model-0a1747-kserve-754bf56797-lfqd6\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.297581 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.297496 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-dshm\") pod \"test-model-0a1747-kserve-754bf56797-lfqd6\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.297581 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.297574 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-home\") pod \"test-model-0a1747-kserve-754bf56797-lfqd6\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.297654 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.297594 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-tmp-dir\") pod \"test-model-0a1747-kserve-754bf56797-lfqd6\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.297654 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.297623 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7cqp\" (UniqueName: \"kubernetes.io/projected/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-kube-api-access-j7cqp\") pod \"test-model-0a1747-kserve-754bf56797-lfqd6\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.297730 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.297665 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-tls-certs\") pod \"test-model-0a1747-kserve-754bf56797-lfqd6\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.398676 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.398643 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-tls-certs\") pod \"test-model-0a1747-kserve-754bf56797-lfqd6\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.398884 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.398699 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-model-cache\") pod \"test-model-0a1747-kserve-754bf56797-lfqd6\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.398884 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.398736 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-dshm\") pod \"test-model-0a1747-kserve-754bf56797-lfqd6\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.398884 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.398818 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-home\") pod \"test-model-0a1747-kserve-754bf56797-lfqd6\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.398884 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.398841 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-tmp-dir\") pod \"test-model-0a1747-kserve-754bf56797-lfqd6\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.398884 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.398861 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7cqp\" (UniqueName: \"kubernetes.io/projected/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-kube-api-access-j7cqp\") pod \"test-model-0a1747-kserve-754bf56797-lfqd6\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.399197 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.399174 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-model-cache\") pod \"test-model-0a1747-kserve-754bf56797-lfqd6\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.399277 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.399218 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-home\") pod \"test-model-0a1747-kserve-754bf56797-lfqd6\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.399277 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.399258 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-tmp-dir\") pod \"test-model-0a1747-kserve-754bf56797-lfqd6\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.401097 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.401074 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-dshm\") pod \"test-model-0a1747-kserve-754bf56797-lfqd6\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.401206 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.401118 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-tls-certs\") pod \"test-model-0a1747-kserve-754bf56797-lfqd6\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.410237 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.410215 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7cqp\" (UniqueName: \"kubernetes.io/projected/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-kube-api-access-j7cqp\") pod \"test-model-0a1747-kserve-754bf56797-lfqd6\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.558109 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.558069 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:36.690773 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:36.690734 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6"] Jul 13 19:34:36.693807 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:34:36.693764 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25a48c2e_8ed4_49ed_a379_c9b6b3d6c21e.slice/crio-b9f8308fa0c0aa829e272b086969c2a855e98c01ac86be2cfb986788a3f214c1 WatchSource:0}: Error finding container b9f8308fa0c0aa829e272b086969c2a855e98c01ac86be2cfb986788a3f214c1: Status 404 returned error can't find the container with id b9f8308fa0c0aa829e272b086969c2a855e98c01ac86be2cfb986788a3f214c1 Jul 13 19:34:37.636341 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:37.636305 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" event={"ID":"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e","Type":"ContainerStarted","Data":"80d0ccde7c9176b4cdf9317937a6672948ab9ba230e2c67ceef4dbdab5fa4c21"} Jul 13 19:34:37.636341 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:37.636348 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" event={"ID":"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e","Type":"ContainerStarted","Data":"b9f8308fa0c0aa829e272b086969c2a855e98c01ac86be2cfb986788a3f214c1"} Jul 13 19:34:37.654258 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:37.654205 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" podStartSLOduration=1.654190357 podStartE2EDuration="1.654190357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:34:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:34:37.651965304 +0000 UTC m=+2163.128419551" watchObservedRunningTime="2026-07-13 19:34:37.654190357 +0000 UTC m=+2163.130644531" Jul 13 19:34:44.658071 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:44.658040 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:34:46.559004 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:46.558966 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:46.559506 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:46.559041 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:46.571303 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:46.571265 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:46.691594 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:46.691567 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:34:46.741495 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:46.741464 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:46.794178 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:46.794147 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:56.254403 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:56.254371 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426"] Jul 13 19:34:56.273588 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:56.273557 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:57.335668 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:57.335635 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6"] Jul 13 19:34:57.358399 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:57.358366 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:58.018358 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:58.018321 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:34:58.124053 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:34:58.124023 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:04.342776 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:04.342739 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:35:07.965078 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:07.965045 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:11.275534 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.275493 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" podUID="b16c2793-2168-4ef7-a249-438d1fd2be50" containerName="main" containerID="cri-o://e10e46356b3cf2d3e6a9a426fe959f426fd61a87b617073b57643a8e3b3aa623" gracePeriod=45 Jul 13 19:35:11.521447 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.521421 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:35:11.537043 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.536983 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glz9z\" (UniqueName: \"kubernetes.io/projected/b16c2793-2168-4ef7-a249-438d1fd2be50-kube-api-access-glz9z\") pod \"b16c2793-2168-4ef7-a249-438d1fd2be50\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " Jul 13 19:35:11.537043 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.537018 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-model-cache\") pod \"b16c2793-2168-4ef7-a249-438d1fd2be50\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " Jul 13 19:35:11.537043 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.537038 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b16c2793-2168-4ef7-a249-438d1fd2be50-tls-certs\") pod \"b16c2793-2168-4ef7-a249-438d1fd2be50\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " Jul 13 19:35:11.537299 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.537063 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-tmp-dir\") pod \"b16c2793-2168-4ef7-a249-438d1fd2be50\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " Jul 13 19:35:11.537299 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.537083 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-home\") pod \"b16c2793-2168-4ef7-a249-438d1fd2be50\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " Jul 13 19:35:11.537299 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.537151 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-dshm\") pod \"b16c2793-2168-4ef7-a249-438d1fd2be50\" (UID: \"b16c2793-2168-4ef7-a249-438d1fd2be50\") " Jul 13 19:35:11.537459 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.537314 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-model-cache" (OuterVolumeSpecName: "model-cache") pod "b16c2793-2168-4ef7-a249-438d1fd2be50" (UID: "b16c2793-2168-4ef7-a249-438d1fd2be50"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:35:11.537459 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.537331 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b16c2793-2168-4ef7-a249-438d1fd2be50" (UID: "b16c2793-2168-4ef7-a249-438d1fd2be50"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:35:11.537459 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.537388 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-home" (OuterVolumeSpecName: "home") pod "b16c2793-2168-4ef7-a249-438d1fd2be50" (UID: "b16c2793-2168-4ef7-a249-438d1fd2be50"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:35:11.537618 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.537533 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:35:11.537618 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.537552 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:35:11.537618 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.537564 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:35:11.539372 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.539343 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b16c2793-2168-4ef7-a249-438d1fd2be50-kube-api-access-glz9z" (OuterVolumeSpecName: "kube-api-access-glz9z") pod "b16c2793-2168-4ef7-a249-438d1fd2be50" (UID: "b16c2793-2168-4ef7-a249-438d1fd2be50"). InnerVolumeSpecName "kube-api-access-glz9z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:35:11.539483 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.539368 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-dshm" (OuterVolumeSpecName: "dshm") pod "b16c2793-2168-4ef7-a249-438d1fd2be50" (UID: "b16c2793-2168-4ef7-a249-438d1fd2be50"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:35:11.539483 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.539371 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b16c2793-2168-4ef7-a249-438d1fd2be50-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b16c2793-2168-4ef7-a249-438d1fd2be50" (UID: "b16c2793-2168-4ef7-a249-438d1fd2be50"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:35:11.638245 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.638208 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b16c2793-2168-4ef7-a249-438d1fd2be50-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:35:11.638245 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.638240 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glz9z\" (UniqueName: \"kubernetes.io/projected/b16c2793-2168-4ef7-a249-438d1fd2be50-kube-api-access-glz9z\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:35:11.638245 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.638250 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b16c2793-2168-4ef7-a249-438d1fd2be50-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:35:11.798928 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.798896 2574 generic.go:356] "Generic (PLEG): container finished" podID="b16c2793-2168-4ef7-a249-438d1fd2be50" containerID="e10e46356b3cf2d3e6a9a426fe959f426fd61a87b617073b57643a8e3b3aa623" exitCode=0 Jul 13 19:35:11.799087 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.798933 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" Jul 13 19:35:11.799087 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.798982 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" event={"ID":"b16c2793-2168-4ef7-a249-438d1fd2be50","Type":"ContainerDied","Data":"e10e46356b3cf2d3e6a9a426fe959f426fd61a87b617073b57643a8e3b3aa623"} Jul 13 19:35:11.799087 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.799026 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426" event={"ID":"b16c2793-2168-4ef7-a249-438d1fd2be50","Type":"ContainerDied","Data":"d07382194e09c2a3dabe81d52d7663d2df80bf52e9c4fed5ef8b40e11eef2359"} Jul 13 19:35:11.799087 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.799042 2574 scope.go:117] "RemoveContainer" containerID="e10e46356b3cf2d3e6a9a426fe959f426fd61a87b617073b57643a8e3b3aa623" Jul 13 19:35:11.808001 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.807987 2574 scope.go:117] "RemoveContainer" containerID="e10e46356b3cf2d3e6a9a426fe959f426fd61a87b617073b57643a8e3b3aa623" Jul 13 19:35:11.808297 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:35:11.808279 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e10e46356b3cf2d3e6a9a426fe959f426fd61a87b617073b57643a8e3b3aa623\": container with ID starting with e10e46356b3cf2d3e6a9a426fe959f426fd61a87b617073b57643a8e3b3aa623 not found: ID does not exist" containerID="e10e46356b3cf2d3e6a9a426fe959f426fd61a87b617073b57643a8e3b3aa623" Jul 13 19:35:11.808366 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.808305 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e10e46356b3cf2d3e6a9a426fe959f426fd61a87b617073b57643a8e3b3aa623"} err="failed to get container status \"e10e46356b3cf2d3e6a9a426fe959f426fd61a87b617073b57643a8e3b3aa623\": rpc error: code = NotFound desc = could not find container \"e10e46356b3cf2d3e6a9a426fe959f426fd61a87b617073b57643a8e3b3aa623\": container with ID starting with e10e46356b3cf2d3e6a9a426fe959f426fd61a87b617073b57643a8e3b3aa623 not found: ID does not exist" Jul 13 19:35:11.823679 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.823652 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426"] Jul 13 19:35:11.826172 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:11.826147 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-d0bb2f/test-model-d0bb2f-kserve-6bdccddd77-df426"] Jul 13 19:35:12.356533 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.356462 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" podUID="25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e" containerName="main" containerID="cri-o://80d0ccde7c9176b4cdf9317937a6672948ab9ba230e2c67ceef4dbdab5fa4c21" gracePeriod=45 Jul 13 19:35:12.558645 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.558612 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:12.613611 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.613543 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:35:12.647159 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.647129 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-home\") pod \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " Jul 13 19:35:12.647351 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.647204 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-tmp-dir\") pod \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " Jul 13 19:35:12.647351 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.647254 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7cqp\" (UniqueName: \"kubernetes.io/projected/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-kube-api-access-j7cqp\") pod \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " Jul 13 19:35:12.647351 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.647291 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-model-cache\") pod \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " Jul 13 19:35:12.647548 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.647353 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-tls-certs\") pod \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " Jul 13 19:35:12.647548 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.647377 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-home" (OuterVolumeSpecName: "home") pod "25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e" (UID: "25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:35:12.647548 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.647439 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-dshm\") pod \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\" (UID: \"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e\") " Jul 13 19:35:12.647548 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.647532 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e" (UID: "25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:35:12.647741 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.647569 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-model-cache" (OuterVolumeSpecName: "model-cache") pod "25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e" (UID: "25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:35:12.647834 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.647759 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:35:12.647834 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.647772 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:35:12.647834 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.647814 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:35:12.649595 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.649571 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-dshm" (OuterVolumeSpecName: "dshm") pod "25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e" (UID: "25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:35:12.649705 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.649621 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e" (UID: "25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:35:12.649813 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.649761 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-kube-api-access-j7cqp" (OuterVolumeSpecName: "kube-api-access-j7cqp") pod "25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e" (UID: "25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e"). InnerVolumeSpecName "kube-api-access-j7cqp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:35:12.748587 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.748526 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:35:12.748587 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.748581 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7cqp\" (UniqueName: \"kubernetes.io/projected/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-kube-api-access-j7cqp\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:35:12.748587 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.748592 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:35:12.806402 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.806364 2574 generic.go:356] "Generic (PLEG): container finished" podID="25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e" containerID="80d0ccde7c9176b4cdf9317937a6672948ab9ba230e2c67ceef4dbdab5fa4c21" exitCode=0 Jul 13 19:35:12.806572 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.806408 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" Jul 13 19:35:12.806572 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.806450 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" event={"ID":"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e","Type":"ContainerDied","Data":"80d0ccde7c9176b4cdf9317937a6672948ab9ba230e2c67ceef4dbdab5fa4c21"} Jul 13 19:35:12.806572 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.806493 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6" event={"ID":"25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e","Type":"ContainerDied","Data":"b9f8308fa0c0aa829e272b086969c2a855e98c01ac86be2cfb986788a3f214c1"} Jul 13 19:35:12.806572 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.806511 2574 scope.go:117] "RemoveContainer" containerID="80d0ccde7c9176b4cdf9317937a6672948ab9ba230e2c67ceef4dbdab5fa4c21" Jul 13 19:35:12.822325 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.822307 2574 scope.go:117] "RemoveContainer" containerID="80d0ccde7c9176b4cdf9317937a6672948ab9ba230e2c67ceef4dbdab5fa4c21" Jul 13 19:35:12.822635 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:35:12.822610 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80d0ccde7c9176b4cdf9317937a6672948ab9ba230e2c67ceef4dbdab5fa4c21\": container with ID starting with 80d0ccde7c9176b4cdf9317937a6672948ab9ba230e2c67ceef4dbdab5fa4c21 not found: ID does not exist" containerID="80d0ccde7c9176b4cdf9317937a6672948ab9ba230e2c67ceef4dbdab5fa4c21" Jul 13 19:35:12.822719 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.822647 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80d0ccde7c9176b4cdf9317937a6672948ab9ba230e2c67ceef4dbdab5fa4c21"} err="failed to get container status \"80d0ccde7c9176b4cdf9317937a6672948ab9ba230e2c67ceef4dbdab5fa4c21\": rpc error: code = NotFound desc = could not find container \"80d0ccde7c9176b4cdf9317937a6672948ab9ba230e2c67ceef4dbdab5fa4c21\": container with ID starting with 80d0ccde7c9176b4cdf9317937a6672948ab9ba230e2c67ceef4dbdab5fa4c21 not found: ID does not exist" Jul 13 19:35:12.832391 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.832364 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6"] Jul 13 19:35:12.835903 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:12.835878 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-0a1747/test-model-0a1747-kserve-754bf56797-lfqd6"] Jul 13 19:35:13.054425 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:13.054389 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e" path="/var/lib/kubelet/pods/25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e/volumes" Jul 13 19:35:13.054852 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:13.054838 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b16c2793-2168-4ef7-a249-438d1fd2be50" path="/var/lib/kubelet/pods/b16c2793-2168-4ef7-a249-438d1fd2be50/volumes" Jul 13 19:35:18.426896 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:18.426860 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:18.527054 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:18.527021 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:19.351962 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:19.351929 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:35:23.322941 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:23.322907 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:23.666435 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:23.666349 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f"] Jul 13 19:35:23.666750 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:23.666737 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e" containerName="main" Jul 13 19:35:23.666825 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:23.666751 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e" containerName="main" Jul 13 19:35:23.666825 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:23.666763 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b16c2793-2168-4ef7-a249-438d1fd2be50" containerName="main" Jul 13 19:35:23.666825 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:23.666769 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b16c2793-2168-4ef7-a249-438d1fd2be50" containerName="main" Jul 13 19:35:23.666953 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:23.666839 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="25a48c2e-8ed4-49ed-a379-c9b6b3d6c21e" containerName="main" Jul 13 19:35:23.666953 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:23.666849 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b16c2793-2168-4ef7-a249-438d1fd2be50" containerName="main" Jul 13 19:35:23.671379 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:23.671358 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f" Jul 13 19:35:23.674273 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:23.674245 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-llvfh\"" Jul 13 19:35:23.674464 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:23.674448 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 19:35:23.683740 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:23.683712 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f"] Jul 13 19:35:23.754421 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:23.754381 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/315a13c8-8080-443d-a63a-4b60636c5a42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f\" (UID: \"315a13c8-8080-443d-a63a-4b60636c5a42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f" Jul 13 19:35:23.855394 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:23.855359 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/315a13c8-8080-443d-a63a-4b60636c5a42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f\" (UID: \"315a13c8-8080-443d-a63a-4b60636c5a42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f" Jul 13 19:35:23.856012 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:23.855993 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/315a13c8-8080-443d-a63a-4b60636c5a42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f\" (UID: \"315a13c8-8080-443d-a63a-4b60636c5a42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f" Jul 13 19:35:23.981437 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:23.981336 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f" Jul 13 19:35:24.114160 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:24.114137 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f"] Jul 13 19:35:24.116286 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:35:24.116261 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod315a13c8_8080_443d_a63a_4b60636c5a42.slice/crio-7124b3ea8ae2cbc47899a704c79acf3b110a5a34fa5f5cc247f7bc0e05471630 WatchSource:0}: Error finding container 7124b3ea8ae2cbc47899a704c79acf3b110a5a34fa5f5cc247f7bc0e05471630: Status 404 returned error can't find the container with id 7124b3ea8ae2cbc47899a704c79acf3b110a5a34fa5f5cc247f7bc0e05471630 Jul 13 19:35:24.118714 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:24.118699 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 19:35:24.867970 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:24.867930 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f" event={"ID":"315a13c8-8080-443d-a63a-4b60636c5a42","Type":"ContainerStarted","Data":"7124b3ea8ae2cbc47899a704c79acf3b110a5a34fa5f5cc247f7bc0e05471630"} Jul 13 19:35:25.084837 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:25.084774 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:25.342877 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:25.342843 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:26.882200 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:26.882145 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f" event={"ID":"315a13c8-8080-443d-a63a-4b60636c5a42","Type":"ContainerStarted","Data":"c1212df1bff662c83b0e8360e3a249d657018e83669a17a47b5709273b4d40a2"} Jul 13 19:35:26.896799 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:26.896733 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f" podStartSLOduration=1.550078187 podStartE2EDuration="3.896717133s" totalImagesPullingTime="2.346638946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:35:26.895540161 +0000 UTC m=+2212.371994388" watchObservedRunningTime="2026-07-13 19:35:26.896717133 +0000 UTC m=+2212.373171306" Jul 13 19:35:27.889382 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:27.889343 2574 generic.go:356] "Generic (PLEG): container finished" podID="315a13c8-8080-443d-a63a-4b60636c5a42" containerID="c1212df1bff662c83b0e8360e3a249d657018e83669a17a47b5709273b4d40a2" exitCode=0 Jul 13 19:35:27.889738 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:27.889379 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f" event={"ID":"315a13c8-8080-443d-a63a-4b60636c5a42","Type":"ContainerDied","Data":"c1212df1bff662c83b0e8360e3a249d657018e83669a17a47b5709273b4d40a2"} Jul 13 19:35:29.023847 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:29.023821 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f" Jul 13 19:35:29.105121 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:29.105083 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/315a13c8-8080-443d-a63a-4b60636c5a42-maas-api-service-ca\") pod \"315a13c8-8080-443d-a63a-4b60636c5a42\" (UID: \"315a13c8-8080-443d-a63a-4b60636c5a42\") " Jul 13 19:35:29.105471 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:29.105444 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/315a13c8-8080-443d-a63a-4b60636c5a42-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "315a13c8-8080-443d-a63a-4b60636c5a42" (UID: "315a13c8-8080-443d-a63a-4b60636c5a42"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:35:29.206376 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:29.206300 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/315a13c8-8080-443d-a63a-4b60636c5a42-maas-api-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:35:29.901779 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:29.901750 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f" Jul 13 19:35:29.901990 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:29.901809 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f" event={"ID":"315a13c8-8080-443d-a63a-4b60636c5a42","Type":"ContainerDied","Data":"7124b3ea8ae2cbc47899a704c79acf3b110a5a34fa5f5cc247f7bc0e05471630"} Jul 13 19:35:29.901990 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:29.901858 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7124b3ea8ae2cbc47899a704c79acf3b110a5a34fa5f5cc247f7bc0e05471630" Jul 13 19:35:34.221205 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:34.221168 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:37.449142 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:37.449105 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:35:38.767234 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:38.767202 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:45.038844 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:45.038802 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:45.103828 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:45.103795 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:49.454173 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:49.454133 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:49.832210 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:49.832166 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:50.048892 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:50.048851 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:50.387099 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:50.387061 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:50.720903 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:50.720812 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:52.873642 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:52.873603 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:52.974150 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:52.974109 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:53.085120 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:53.085086 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:53.650065 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:53.650032 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:53.749999 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:53.749968 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:53.947525 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:53.947439 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:57.364399 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:57.364323 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:57.820434 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:57.820382 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:58.124080 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:58.123993 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:35:58.252972 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:35:58.252935 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:00.960795 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:00.960754 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:01.055496 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:01.055464 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:01.214069 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:01.213987 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:01.258185 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:01.258151 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:36:01.648290 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:01.648259 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:01.746568 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:01.746534 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:01.961588 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:01.961500 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:05.467227 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:05.467184 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:13.481885 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:13.481850 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-868895-8nrlk"] Jul 13 19:36:13.482269 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:13.482228 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="315a13c8-8080-443d-a63a-4b60636c5a42" containerName="revoke-keys" Jul 13 19:36:13.482269 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:13.482240 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="315a13c8-8080-443d-a63a-4b60636c5a42" containerName="revoke-keys" Jul 13 19:36:13.482342 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:13.482321 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="315a13c8-8080-443d-a63a-4b60636c5a42" containerName="revoke-keys" Jul 13 19:36:13.485254 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:13.485239 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-868895-8nrlk" Jul 13 19:36:13.487573 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:13.487547 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-llvfh\"" Jul 13 19:36:13.487726 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:13.487590 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 19:36:13.491494 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:13.491470 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-868895-8nrlk"] Jul 13 19:36:13.628030 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:13.627995 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7582a728-a2cc-4f02-adac-8c0d3f0158f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-868895-8nrlk\" (UID: \"7582a728-a2cc-4f02-adac-8c0d3f0158f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-868895-8nrlk" Jul 13 19:36:13.729125 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:13.729088 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7582a728-a2cc-4f02-adac-8c0d3f0158f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-868895-8nrlk\" (UID: \"7582a728-a2cc-4f02-adac-8c0d3f0158f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-868895-8nrlk" Jul 13 19:36:13.729824 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:13.729773 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7582a728-a2cc-4f02-adac-8c0d3f0158f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-868895-8nrlk\" (UID: \"7582a728-a2cc-4f02-adac-8c0d3f0158f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-868895-8nrlk" Jul 13 19:36:13.797305 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:13.797267 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-868895-8nrlk" Jul 13 19:36:13.931914 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:13.931885 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-868895-8nrlk"] Jul 13 19:36:13.933582 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:36:13.933555 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7582a728_a2cc_4f02_adac_8c0d3f0158f0.slice/crio-94f62d33c7b93f1837096a269bec1aa5f5bb73558e43f0303dbbfdd97ffb2547 WatchSource:0}: Error finding container 94f62d33c7b93f1837096a269bec1aa5f5bb73558e43f0303dbbfdd97ffb2547: Status 404 returned error can't find the container with id 94f62d33c7b93f1837096a269bec1aa5f5bb73558e43f0303dbbfdd97ffb2547 Jul 13 19:36:14.117856 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:14.117818 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-868895-8nrlk" event={"ID":"7582a728-a2cc-4f02-adac-8c0d3f0158f0","Type":"ContainerStarted","Data":"94f62d33c7b93f1837096a269bec1aa5f5bb73558e43f0303dbbfdd97ffb2547"} Jul 13 19:36:14.336778 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:14.336690 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:14.437147 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:14.437118 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:15.124215 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:15.124185 2574 generic.go:356] "Generic (PLEG): container finished" podID="7582a728-a2cc-4f02-adac-8c0d3f0158f0" containerID="4991a2876e6c098b949dd4569fecd8b5d1d9665fa02112e8f52d4f33c7697169" exitCode=0 Jul 13 19:36:15.124621 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:15.124219 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-868895-8nrlk" event={"ID":"7582a728-a2cc-4f02-adac-8c0d3f0158f0","Type":"ContainerDied","Data":"4991a2876e6c098b949dd4569fecd8b5d1d9665fa02112e8f52d4f33c7697169"} Jul 13 19:36:16.258412 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:16.258387 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-868895-8nrlk" Jul 13 19:36:16.356948 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:16.356913 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7582a728-a2cc-4f02-adac-8c0d3f0158f0-maas-api-service-ca\") pod \"7582a728-a2cc-4f02-adac-8c0d3f0158f0\" (UID: \"7582a728-a2cc-4f02-adac-8c0d3f0158f0\") " Jul 13 19:36:16.357259 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:16.357236 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7582a728-a2cc-4f02-adac-8c0d3f0158f0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7582a728-a2cc-4f02-adac-8c0d3f0158f0" (UID: "7582a728-a2cc-4f02-adac-8c0d3f0158f0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:36:16.458299 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:16.458206 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7582a728-a2cc-4f02-adac-8c0d3f0158f0-maas-api-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:36:17.135021 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:17.134988 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-868895-8nrlk" event={"ID":"7582a728-a2cc-4f02-adac-8c0d3f0158f0","Type":"ContainerDied","Data":"94f62d33c7b93f1837096a269bec1aa5f5bb73558e43f0303dbbfdd97ffb2547"} Jul 13 19:36:17.135021 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:17.135020 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-868895-8nrlk" Jul 13 19:36:17.135244 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:17.135039 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="94f62d33c7b93f1837096a269bec1aa5f5bb73558e43f0303dbbfdd97ffb2547" Jul 13 19:36:18.939954 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:18.939922 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:36:22.717681 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:22.717642 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:22.818381 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:22.818342 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:24.218974 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:24.218942 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:24.320350 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:24.320315 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:24.497433 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:24.497343 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:28.989438 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:28.989402 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:29.709411 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:29.709367 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:31.246118 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:31.246075 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:36:34.588711 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:34.588674 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:34.649559 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:34.649518 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:39.258561 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:39.258525 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:39.494396 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:39.494358 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:39.613455 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:39.613423 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f2d389-7rnjp"] Jul 13 19:36:39.613836 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:39.613823 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7582a728-a2cc-4f02-adac-8c0d3f0158f0" containerName="revoke-keys" Jul 13 19:36:39.613901 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:39.613838 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7582a728-a2cc-4f02-adac-8c0d3f0158f0" containerName="revoke-keys" Jul 13 19:36:39.613939 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:39.613903 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7582a728-a2cc-4f02-adac-8c0d3f0158f0" containerName="revoke-keys" Jul 13 19:36:39.617242 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:39.617226 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f2d389-7rnjp" Jul 13 19:36:39.619398 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:39.619373 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 19:36:39.619515 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:39.619439 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-llvfh\"" Jul 13 19:36:39.623108 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:39.623087 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f2d389-7rnjp"] Jul 13 19:36:39.674897 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:39.674865 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61dc57b6-95dc-4ce1-9102-11d1649db336-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f2d389-7rnjp\" (UID: \"61dc57b6-95dc-4ce1-9102-11d1649db336\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f2d389-7rnjp" Jul 13 19:36:39.775513 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:39.775470 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61dc57b6-95dc-4ce1-9102-11d1649db336-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f2d389-7rnjp\" (UID: \"61dc57b6-95dc-4ce1-9102-11d1649db336\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f2d389-7rnjp" Jul 13 19:36:39.776160 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:39.776141 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61dc57b6-95dc-4ce1-9102-11d1649db336-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f2d389-7rnjp\" (UID: \"61dc57b6-95dc-4ce1-9102-11d1649db336\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f2d389-7rnjp" Jul 13 19:36:39.928825 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:39.928721 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f2d389-7rnjp" Jul 13 19:36:40.062709 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:40.062683 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f2d389-7rnjp"] Jul 13 19:36:40.064531 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:36:40.064498 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61dc57b6_95dc_4ce1_9102_11d1649db336.slice/crio-0c77dd6ebd75291470fb9f4865b05a4359d620c6dd78fd16b64c5213d879ef2a WatchSource:0}: Error finding container 0c77dd6ebd75291470fb9f4865b05a4359d620c6dd78fd16b64c5213d879ef2a: Status 404 returned error can't find the container with id 0c77dd6ebd75291470fb9f4865b05a4359d620c6dd78fd16b64c5213d879ef2a Jul 13 19:36:40.254052 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:40.254022 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f2d389-7rnjp" event={"ID":"61dc57b6-95dc-4ce1-9102-11d1649db336","Type":"ContainerStarted","Data":"0c77dd6ebd75291470fb9f4865b05a4359d620c6dd78fd16b64c5213d879ef2a"} Jul 13 19:36:40.274288 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:40.274240 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:40.375196 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:40.375156 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:41.261021 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:41.260986 2574 generic.go:356] "Generic (PLEG): container finished" podID="61dc57b6-95dc-4ce1-9102-11d1649db336" containerID="68b51683411eb3a4edfa33d55f3458464db484153e6c263c05489b4264467f97" exitCode=0 Jul 13 19:36:41.261021 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:41.261023 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f2d389-7rnjp" event={"ID":"61dc57b6-95dc-4ce1-9102-11d1649db336","Type":"ContainerDied","Data":"68b51683411eb3a4edfa33d55f3458464db484153e6c263c05489b4264467f97"} Jul 13 19:36:42.400804 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:42.400750 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f2d389-7rnjp" Jul 13 19:36:42.502031 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:42.501999 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61dc57b6-95dc-4ce1-9102-11d1649db336-maas-api-service-ca\") pod \"61dc57b6-95dc-4ce1-9102-11d1649db336\" (UID: \"61dc57b6-95dc-4ce1-9102-11d1649db336\") " Jul 13 19:36:42.502398 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:42.502378 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61dc57b6-95dc-4ce1-9102-11d1649db336-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "61dc57b6-95dc-4ce1-9102-11d1649db336" (UID: "61dc57b6-95dc-4ce1-9102-11d1649db336"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:36:42.603040 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:42.603002 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61dc57b6-95dc-4ce1-9102-11d1649db336-maas-api-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:36:43.271816 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:43.271768 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f2d389-7rnjp" event={"ID":"61dc57b6-95dc-4ce1-9102-11d1649db336","Type":"ContainerDied","Data":"0c77dd6ebd75291470fb9f4865b05a4359d620c6dd78fd16b64c5213d879ef2a"} Jul 13 19:36:43.271816 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:43.271813 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f2d389-7rnjp" Jul 13 19:36:43.272024 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:43.271834 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c77dd6ebd75291470fb9f4865b05a4359d620c6dd78fd16b64c5213d879ef2a" Jul 13 19:36:50.080283 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:50.080250 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:53.541763 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:53.541728 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:36:55.135520 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:55.135484 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:59.865999 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:59.865961 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:36:59.965060 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:36:59.965025 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:00.805596 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:00.805563 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:01.092266 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:01.092182 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:05.387508 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:05.387475 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:07.293126 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:07.293091 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:09.363377 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:09.363340 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:09.451549 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:09.451518 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:09.528961 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:09.528925 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:09.626335 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:09.626250 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:10.120920 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:10.120889 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:10.226977 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:10.226939 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:12.248173 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:12.248133 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:37:13.865336 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:13.865299 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:14.199092 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:14.198971 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:14.515092 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:14.514998 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:14.763385 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:14.763351 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:17.418025 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:17.417989 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:17.487627 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:17.487593 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:17.637351 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:17.637318 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:18.002057 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:18.002015 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:18.196968 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:18.196931 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:18.294538 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:18.294507 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:18.546777 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:18.546662 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:21.934653 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:21.934621 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:30.907879 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:30.907778 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:31.007669 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:31.007628 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:32.948819 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:32.948765 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:37:34.535281 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.535253 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb"] Jul 13 19:37:34.535666 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.535639 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61dc57b6-95dc-4ce1-9102-11d1649db336" containerName="revoke-keys" Jul 13 19:37:34.535666 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.535651 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="61dc57b6-95dc-4ce1-9102-11d1649db336" containerName="revoke-keys" Jul 13 19:37:34.535776 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.535715 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="61dc57b6-95dc-4ce1-9102-11d1649db336" containerName="revoke-keys" Jul 13 19:37:34.539499 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.539478 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.541776 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.541753 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-196ed1\"/\"kube-root-ca.crt\"" Jul 13 19:37:34.541942 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.541829 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-196ed1\"/\"openshift-service-ca.crt\"" Jul 13 19:37:34.542588 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.542566 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-196ed1\"/\"auth-test-model-196ed1-kserve-self-signed-certs\"" Jul 13 19:37:34.542821 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.542569 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-196ed1\"/\"default-dockercfg-x8ddf\"" Jul 13 19:37:34.553689 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.551674 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb"] Jul 13 19:37:34.575269 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.575231 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:34.703567 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.703533 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-tmp-dir\") pod \"auth-test-model-196ed1-kserve-78998bd8f5-6xwsb\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.703567 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.703568 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-model-cache\") pod \"auth-test-model-196ed1-kserve-78998bd8f5-6xwsb\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.703857 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.703657 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2wz8\" (UniqueName: \"kubernetes.io/projected/9c6362b7-c77d-4995-b2a0-dfe75e22edba-kube-api-access-s2wz8\") pod \"auth-test-model-196ed1-kserve-78998bd8f5-6xwsb\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.703857 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.703835 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c6362b7-c77d-4995-b2a0-dfe75e22edba-tls-certs\") pod \"auth-test-model-196ed1-kserve-78998bd8f5-6xwsb\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.703973 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.703869 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-dshm\") pod \"auth-test-model-196ed1-kserve-78998bd8f5-6xwsb\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.703973 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.703894 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-home\") pod \"auth-test-model-196ed1-kserve-78998bd8f5-6xwsb\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.804718 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.804683 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-tmp-dir\") pod \"auth-test-model-196ed1-kserve-78998bd8f5-6xwsb\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.804718 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.804727 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-model-cache\") pod \"auth-test-model-196ed1-kserve-78998bd8f5-6xwsb\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.805004 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.804797 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2wz8\" (UniqueName: \"kubernetes.io/projected/9c6362b7-c77d-4995-b2a0-dfe75e22edba-kube-api-access-s2wz8\") pod \"auth-test-model-196ed1-kserve-78998bd8f5-6xwsb\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.805004 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.804901 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c6362b7-c77d-4995-b2a0-dfe75e22edba-tls-certs\") pod \"auth-test-model-196ed1-kserve-78998bd8f5-6xwsb\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.805004 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.804932 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-dshm\") pod \"auth-test-model-196ed1-kserve-78998bd8f5-6xwsb\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.805004 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.804956 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-home\") pod \"auth-test-model-196ed1-kserve-78998bd8f5-6xwsb\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.805222 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.805149 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-tmp-dir\") pod \"auth-test-model-196ed1-kserve-78998bd8f5-6xwsb\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.805222 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.805193 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-model-cache\") pod \"auth-test-model-196ed1-kserve-78998bd8f5-6xwsb\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.805326 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.805265 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-home\") pod \"auth-test-model-196ed1-kserve-78998bd8f5-6xwsb\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.807129 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.807101 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-dshm\") pod \"auth-test-model-196ed1-kserve-78998bd8f5-6xwsb\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.807370 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.807353 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c6362b7-c77d-4995-b2a0-dfe75e22edba-tls-certs\") pod \"auth-test-model-196ed1-kserve-78998bd8f5-6xwsb\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.815689 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.815668 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2wz8\" (UniqueName: \"kubernetes.io/projected/9c6362b7-c77d-4995-b2a0-dfe75e22edba-kube-api-access-s2wz8\") pod \"auth-test-model-196ed1-kserve-78998bd8f5-6xwsb\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.856234 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.856206 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:34.992305 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:34.992280 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb"] Jul 13 19:37:34.994578 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:37:34.994554 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c6362b7_c77d_4995_b2a0_dfe75e22edba.slice/crio-0afd857952e640dba47dd4462ea4a8a755eea9065e7c32cebe96756c395641a3 WatchSource:0}: Error finding container 0afd857952e640dba47dd4462ea4a8a755eea9065e7c32cebe96756c395641a3: Status 404 returned error can't find the container with id 0afd857952e640dba47dd4462ea4a8a755eea9065e7c32cebe96756c395641a3 Jul 13 19:37:35.535767 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:35.535730 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" event={"ID":"9c6362b7-c77d-4995-b2a0-dfe75e22edba","Type":"ContainerStarted","Data":"0ea3bee46176dfb788a3c2aba095c4c50eaa76ebe8c2a566f38337de812e6096"} Jul 13 19:37:35.535767 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:35.535769 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" event={"ID":"9c6362b7-c77d-4995-b2a0-dfe75e22edba","Type":"ContainerStarted","Data":"0afd857952e640dba47dd4462ea4a8a755eea9065e7c32cebe96756c395641a3"} Jul 13 19:37:35.554763 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:35.554709 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" podStartSLOduration=1.554692176 podStartE2EDuration="1.554692176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:37:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:37:35.552758584 +0000 UTC m=+2341.029212757" watchObservedRunningTime="2026-07-13 19:37:35.554692176 +0000 UTC m=+2341.031146349" Jul 13 19:37:39.065976 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:39.065944 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:39.167718 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:39.167680 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:37:44.638502 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:44.638467 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:37:44.856865 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:44.856819 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:44.857123 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:44.857100 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:44.869404 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:44.869373 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:37:45.600248 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:37:45.600218 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:38:18.529701 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.529666 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn"] Jul 13 19:38:18.533331 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.533308 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.535740 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.535714 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2555bb\"/\"openshift-service-ca.crt\"" Jul 13 19:38:18.536551 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.536533 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2555bb\"/\"kube-root-ca.crt\"" Jul 13 19:38:18.536622 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.536606 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2555bb\"/\"auth-test-model-2555bb-kserve-self-signed-certs\"" Jul 13 19:38:18.536677 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.536633 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2555bb\"/\"default-dockercfg-5cn5w\"" Jul 13 19:38:18.544074 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.544052 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn"] Jul 13 19:38:18.568637 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.568600 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:38:18.722644 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.722612 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmwfv\" (UniqueName: \"kubernetes.io/projected/c504d470-8b16-4385-84a6-552c854da94a-kube-api-access-pmwfv\") pod \"auth-test-model-2555bb-kserve-85ff5c975f-6cxvn\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.722869 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.722669 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-home\") pod \"auth-test-model-2555bb-kserve-85ff5c975f-6cxvn\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.722869 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.722695 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c504d470-8b16-4385-84a6-552c854da94a-tls-certs\") pod \"auth-test-model-2555bb-kserve-85ff5c975f-6cxvn\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.722869 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.722752 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-model-cache\") pod \"auth-test-model-2555bb-kserve-85ff5c975f-6cxvn\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.722869 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.722820 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-dshm\") pod \"auth-test-model-2555bb-kserve-85ff5c975f-6cxvn\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.723040 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.722882 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-tmp-dir\") pod \"auth-test-model-2555bb-kserve-85ff5c975f-6cxvn\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.824230 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.824191 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-model-cache\") pod \"auth-test-model-2555bb-kserve-85ff5c975f-6cxvn\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.824230 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.824234 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-dshm\") pod \"auth-test-model-2555bb-kserve-85ff5c975f-6cxvn\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.824453 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.824269 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-tmp-dir\") pod \"auth-test-model-2555bb-kserve-85ff5c975f-6cxvn\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.824453 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.824314 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmwfv\" (UniqueName: \"kubernetes.io/projected/c504d470-8b16-4385-84a6-552c854da94a-kube-api-access-pmwfv\") pod \"auth-test-model-2555bb-kserve-85ff5c975f-6cxvn\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.824453 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.824359 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-home\") pod \"auth-test-model-2555bb-kserve-85ff5c975f-6cxvn\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.824453 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.824390 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c504d470-8b16-4385-84a6-552c854da94a-tls-certs\") pod \"auth-test-model-2555bb-kserve-85ff5c975f-6cxvn\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.824685 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.824659 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-model-cache\") pod \"auth-test-model-2555bb-kserve-85ff5c975f-6cxvn\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.824751 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.824687 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-tmp-dir\") pod \"auth-test-model-2555bb-kserve-85ff5c975f-6cxvn\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.824834 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.824748 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-home\") pod \"auth-test-model-2555bb-kserve-85ff5c975f-6cxvn\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.826621 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.826598 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-dshm\") pod \"auth-test-model-2555bb-kserve-85ff5c975f-6cxvn\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.826931 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.826915 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c504d470-8b16-4385-84a6-552c854da94a-tls-certs\") pod \"auth-test-model-2555bb-kserve-85ff5c975f-6cxvn\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.831834 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.831812 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmwfv\" (UniqueName: \"kubernetes.io/projected/c504d470-8b16-4385-84a6-552c854da94a-kube-api-access-pmwfv\") pod \"auth-test-model-2555bb-kserve-85ff5c975f-6cxvn\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.843885 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.843856 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:18.977108 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:18.977077 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn"] Jul 13 19:38:18.979298 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:38:18.979265 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc504d470_8b16_4385_84a6_552c854da94a.slice/crio-64846b6f3d3e5eec56775ee8ef8d7b90f9befec3e0ea03b4d57f20bf49771fa2 WatchSource:0}: Error finding container 64846b6f3d3e5eec56775ee8ef8d7b90f9befec3e0ea03b4d57f20bf49771fa2: Status 404 returned error can't find the container with id 64846b6f3d3e5eec56775ee8ef8d7b90f9befec3e0ea03b4d57f20bf49771fa2 Jul 13 19:38:19.757495 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:19.757462 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" event={"ID":"c504d470-8b16-4385-84a6-552c854da94a","Type":"ContainerStarted","Data":"16b6089a729ad5c2f4875452e613e04f7d4b84336b08ac18cc40cbbecef28ffd"} Jul 13 19:38:19.757495 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:19.757501 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" event={"ID":"c504d470-8b16-4385-84a6-552c854da94a","Type":"ContainerStarted","Data":"64846b6f3d3e5eec56775ee8ef8d7b90f9befec3e0ea03b4d57f20bf49771fa2"} Jul 13 19:38:19.776261 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:19.776208 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" podStartSLOduration=1.7761920610000002 podStartE2EDuration="1.776192061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:38:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:38:19.774058255 +0000 UTC m=+2385.250512458" watchObservedRunningTime="2026-07-13 19:38:19.776192061 +0000 UTC m=+2385.252646235" Jul 13 19:38:22.756389 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:22.756351 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:38:28.844184 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:28.844131 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:28.844630 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:28.844213 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:28.857085 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:28.857060 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:29.818596 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:29.818563 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:38:35.137824 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:35.137777 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 19:38:35.142942 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:35.142920 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 19:38:39.655797 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:39.655742 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:38:58.060978 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:38:58.060894 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:39:13.844760 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:39:13.844721 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:39:35.148375 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:39:35.148342 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:39:54.448289 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:39:54.448247 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:40:14.854653 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:14.854618 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:40:29.627381 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:29.627294 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb"] Jul 13 19:40:30.042710 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:30.042673 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:40:30.735263 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:30.735232 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn"] Jul 13 19:40:30.756159 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:30.756128 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:40:31.644901 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:31.644870 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:40:37.753668 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:37.753618 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:40:44.647660 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:44.647593 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" podUID="9c6362b7-c77d-4995-b2a0-dfe75e22edba" containerName="main" containerID="cri-o://0ea3bee46176dfb788a3c2aba095c4c50eaa76ebe8c2a566f38337de812e6096" gracePeriod=45 Jul 13 19:40:44.887243 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:44.887217 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:40:44.950421 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:44.950338 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-model-cache\") pod \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " Jul 13 19:40:44.950421 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:44.950391 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2wz8\" (UniqueName: \"kubernetes.io/projected/9c6362b7-c77d-4995-b2a0-dfe75e22edba-kube-api-access-s2wz8\") pod \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " Jul 13 19:40:44.950421 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:44.950412 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-home\") pod \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " Jul 13 19:40:44.950701 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:44.950450 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c6362b7-c77d-4995-b2a0-dfe75e22edba-tls-certs\") pod \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " Jul 13 19:40:44.950701 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:44.950470 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-dshm\") pod \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " Jul 13 19:40:44.950701 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:44.950504 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-tmp-dir\") pod \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\" (UID: \"9c6362b7-c77d-4995-b2a0-dfe75e22edba\") " Jul 13 19:40:44.950893 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:44.950670 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-model-cache" (OuterVolumeSpecName: "model-cache") pod "9c6362b7-c77d-4995-b2a0-dfe75e22edba" (UID: "9c6362b7-c77d-4995-b2a0-dfe75e22edba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:40:44.950893 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:44.950753 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-home" (OuterVolumeSpecName: "home") pod "9c6362b7-c77d-4995-b2a0-dfe75e22edba" (UID: "9c6362b7-c77d-4995-b2a0-dfe75e22edba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:40:44.950893 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:44.950826 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9c6362b7-c77d-4995-b2a0-dfe75e22edba" (UID: "9c6362b7-c77d-4995-b2a0-dfe75e22edba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:40:44.950893 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:44.950882 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:40:44.951044 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:44.950908 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:40:44.952625 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:44.952595 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-dshm" (OuterVolumeSpecName: "dshm") pod "9c6362b7-c77d-4995-b2a0-dfe75e22edba" (UID: "9c6362b7-c77d-4995-b2a0-dfe75e22edba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:40:44.952625 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:44.952599 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c6362b7-c77d-4995-b2a0-dfe75e22edba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9c6362b7-c77d-4995-b2a0-dfe75e22edba" (UID: "9c6362b7-c77d-4995-b2a0-dfe75e22edba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:40:44.952772 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:44.952713 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c6362b7-c77d-4995-b2a0-dfe75e22edba-kube-api-access-s2wz8" (OuterVolumeSpecName: "kube-api-access-s2wz8") pod "9c6362b7-c77d-4995-b2a0-dfe75e22edba" (UID: "9c6362b7-c77d-4995-b2a0-dfe75e22edba"). InnerVolumeSpecName "kube-api-access-s2wz8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:40:45.051369 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:45.051334 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c6362b7-c77d-4995-b2a0-dfe75e22edba-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:40:45.051369 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:45.051363 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:40:45.051369 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:45.051372 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c6362b7-c77d-4995-b2a0-dfe75e22edba-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:40:45.051592 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:45.051380 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2wz8\" (UniqueName: \"kubernetes.io/projected/9c6362b7-c77d-4995-b2a0-dfe75e22edba-kube-api-access-s2wz8\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:40:45.492687 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:45.492645 2574 generic.go:356] "Generic (PLEG): container finished" podID="9c6362b7-c77d-4995-b2a0-dfe75e22edba" containerID="0ea3bee46176dfb788a3c2aba095c4c50eaa76ebe8c2a566f38337de812e6096" exitCode=0 Jul 13 19:40:45.492887 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:45.492696 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" Jul 13 19:40:45.492887 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:45.492734 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" event={"ID":"9c6362b7-c77d-4995-b2a0-dfe75e22edba","Type":"ContainerDied","Data":"0ea3bee46176dfb788a3c2aba095c4c50eaa76ebe8c2a566f38337de812e6096"} Jul 13 19:40:45.492887 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:45.492796 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb" event={"ID":"9c6362b7-c77d-4995-b2a0-dfe75e22edba","Type":"ContainerDied","Data":"0afd857952e640dba47dd4462ea4a8a755eea9065e7c32cebe96756c395641a3"} Jul 13 19:40:45.492887 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:45.492817 2574 scope.go:117] "RemoveContainer" containerID="0ea3bee46176dfb788a3c2aba095c4c50eaa76ebe8c2a566f38337de812e6096" Jul 13 19:40:45.501627 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:45.501467 2574 scope.go:117] "RemoveContainer" containerID="0ea3bee46176dfb788a3c2aba095c4c50eaa76ebe8c2a566f38337de812e6096" Jul 13 19:40:45.501744 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:40:45.501723 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ea3bee46176dfb788a3c2aba095c4c50eaa76ebe8c2a566f38337de812e6096\": container with ID starting with 0ea3bee46176dfb788a3c2aba095c4c50eaa76ebe8c2a566f38337de812e6096 not found: ID does not exist" containerID="0ea3bee46176dfb788a3c2aba095c4c50eaa76ebe8c2a566f38337de812e6096" Jul 13 19:40:45.501809 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:45.501752 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ea3bee46176dfb788a3c2aba095c4c50eaa76ebe8c2a566f38337de812e6096"} err="failed to get container status \"0ea3bee46176dfb788a3c2aba095c4c50eaa76ebe8c2a566f38337de812e6096\": rpc error: code = NotFound desc = could not find container \"0ea3bee46176dfb788a3c2aba095c4c50eaa76ebe8c2a566f38337de812e6096\": container with ID starting with 0ea3bee46176dfb788a3c2aba095c4c50eaa76ebe8c2a566f38337de812e6096 not found: ID does not exist" Jul 13 19:40:45.508569 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:45.508542 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb"] Jul 13 19:40:45.511617 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:45.511596 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-196ed1/auth-test-model-196ed1-kserve-78998bd8f5-6xwsb"] Jul 13 19:40:45.755739 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:45.755654 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" podUID="c504d470-8b16-4385-84a6-552c854da94a" containerName="main" containerID="cri-o://16b6089a729ad5c2f4875452e613e04f7d4b84336b08ac18cc40cbbecef28ffd" gracePeriod=45 Jul 13 19:40:46.012631 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.012567 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:40:46.059968 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.059930 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:40:46.067508 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.067469 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-tmp-dir\") pod \"c504d470-8b16-4385-84a6-552c854da94a\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " Jul 13 19:40:46.067642 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.067561 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-model-cache\") pod \"c504d470-8b16-4385-84a6-552c854da94a\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " Jul 13 19:40:46.067642 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.067635 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-dshm\") pod \"c504d470-8b16-4385-84a6-552c854da94a\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " Jul 13 19:40:46.067733 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.067662 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c504d470-8b16-4385-84a6-552c854da94a-tls-certs\") pod \"c504d470-8b16-4385-84a6-552c854da94a\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " Jul 13 19:40:46.067733 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.067686 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-home\") pod \"c504d470-8b16-4385-84a6-552c854da94a\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " Jul 13 19:40:46.067863 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.067733 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmwfv\" (UniqueName: \"kubernetes.io/projected/c504d470-8b16-4385-84a6-552c854da94a-kube-api-access-pmwfv\") pod \"c504d470-8b16-4385-84a6-552c854da94a\" (UID: \"c504d470-8b16-4385-84a6-552c854da94a\") " Jul 13 19:40:46.067863 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.067765 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c504d470-8b16-4385-84a6-552c854da94a" (UID: "c504d470-8b16-4385-84a6-552c854da94a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:40:46.067863 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.067795 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-model-cache" (OuterVolumeSpecName: "model-cache") pod "c504d470-8b16-4385-84a6-552c854da94a" (UID: "c504d470-8b16-4385-84a6-552c854da94a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:40:46.068056 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.067990 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-home" (OuterVolumeSpecName: "home") pod "c504d470-8b16-4385-84a6-552c854da94a" (UID: "c504d470-8b16-4385-84a6-552c854da94a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:40:46.068190 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.068169 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:40:46.068246 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.068195 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:40:46.068246 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.068210 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:40:46.070029 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.070003 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c504d470-8b16-4385-84a6-552c854da94a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c504d470-8b16-4385-84a6-552c854da94a" (UID: "c504d470-8b16-4385-84a6-552c854da94a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:40:46.070255 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.070230 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c504d470-8b16-4385-84a6-552c854da94a-kube-api-access-pmwfv" (OuterVolumeSpecName: "kube-api-access-pmwfv") pod "c504d470-8b16-4385-84a6-552c854da94a" (UID: "c504d470-8b16-4385-84a6-552c854da94a"). InnerVolumeSpecName "kube-api-access-pmwfv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:40:46.070255 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.070233 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-dshm" (OuterVolumeSpecName: "dshm") pod "c504d470-8b16-4385-84a6-552c854da94a" (UID: "c504d470-8b16-4385-84a6-552c854da94a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:40:46.168997 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.168960 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c504d470-8b16-4385-84a6-552c854da94a-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:40:46.168997 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.168988 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c504d470-8b16-4385-84a6-552c854da94a-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:40:46.168997 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.168999 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmwfv\" (UniqueName: \"kubernetes.io/projected/c504d470-8b16-4385-84a6-552c854da94a-kube-api-access-pmwfv\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:40:46.499173 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.499138 2574 generic.go:356] "Generic (PLEG): container finished" podID="c504d470-8b16-4385-84a6-552c854da94a" containerID="16b6089a729ad5c2f4875452e613e04f7d4b84336b08ac18cc40cbbecef28ffd" exitCode=0 Jul 13 19:40:46.499364 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.499187 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" Jul 13 19:40:46.499364 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.499226 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" event={"ID":"c504d470-8b16-4385-84a6-552c854da94a","Type":"ContainerDied","Data":"16b6089a729ad5c2f4875452e613e04f7d4b84336b08ac18cc40cbbecef28ffd"} Jul 13 19:40:46.499364 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.499269 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn" event={"ID":"c504d470-8b16-4385-84a6-552c854da94a","Type":"ContainerDied","Data":"64846b6f3d3e5eec56775ee8ef8d7b90f9befec3e0ea03b4d57f20bf49771fa2"} Jul 13 19:40:46.499364 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.499299 2574 scope.go:117] "RemoveContainer" containerID="16b6089a729ad5c2f4875452e613e04f7d4b84336b08ac18cc40cbbecef28ffd" Jul 13 19:40:46.508421 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.508405 2574 scope.go:117] "RemoveContainer" containerID="16b6089a729ad5c2f4875452e613e04f7d4b84336b08ac18cc40cbbecef28ffd" Jul 13 19:40:46.508675 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:40:46.508657 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16b6089a729ad5c2f4875452e613e04f7d4b84336b08ac18cc40cbbecef28ffd\": container with ID starting with 16b6089a729ad5c2f4875452e613e04f7d4b84336b08ac18cc40cbbecef28ffd not found: ID does not exist" containerID="16b6089a729ad5c2f4875452e613e04f7d4b84336b08ac18cc40cbbecef28ffd" Jul 13 19:40:46.508732 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.508687 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16b6089a729ad5c2f4875452e613e04f7d4b84336b08ac18cc40cbbecef28ffd"} err="failed to get container status \"16b6089a729ad5c2f4875452e613e04f7d4b84336b08ac18cc40cbbecef28ffd\": rpc error: code = NotFound desc = could not find container \"16b6089a729ad5c2f4875452e613e04f7d4b84336b08ac18cc40cbbecef28ffd\": container with ID starting with 16b6089a729ad5c2f4875452e613e04f7d4b84336b08ac18cc40cbbecef28ffd not found: ID does not exist" Jul 13 19:40:46.523462 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.523439 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn"] Jul 13 19:40:46.527075 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:46.527053 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2555bb/auth-test-model-2555bb-kserve-85ff5c975f-6cxvn"] Jul 13 19:40:47.055087 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:47.055058 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c6362b7-c77d-4995-b2a0-dfe75e22edba" path="/var/lib/kubelet/pods/9c6362b7-c77d-4995-b2a0-dfe75e22edba/volumes" Jul 13 19:40:47.055472 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:40:47.055458 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c504d470-8b16-4385-84a6-552c854da94a" path="/var/lib/kubelet/pods/c504d470-8b16-4385-84a6-552c854da94a/volumes" Jul 13 19:41:02.963396 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:02.963362 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw"] Jul 13 19:41:02.963862 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:02.963764 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c504d470-8b16-4385-84a6-552c854da94a" containerName="main" Jul 13 19:41:02.963862 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:02.963779 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c504d470-8b16-4385-84a6-552c854da94a" containerName="main" Jul 13 19:41:02.963862 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:02.963819 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c6362b7-c77d-4995-b2a0-dfe75e22edba" containerName="main" Jul 13 19:41:02.963862 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:02.963825 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c6362b7-c77d-4995-b2a0-dfe75e22edba" containerName="main" Jul 13 19:41:02.964008 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:02.963896 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c6362b7-c77d-4995-b2a0-dfe75e22edba" containerName="main" Jul 13 19:41:02.964008 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:02.963905 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c504d470-8b16-4385-84a6-552c854da94a" containerName="main" Jul 13 19:41:02.968355 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:02.968340 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:02.971799 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:02.971760 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2555bb\"/\"default-dockercfg-5cn5w\"" Jul 13 19:41:02.971924 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:02.971834 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2555bb\"/\"kube-root-ca.crt\"" Jul 13 19:41:02.971924 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:02.971834 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2555bb\"/\"sub-test-model-2555bb-kserve-self-signed-certs\"" Jul 13 19:41:02.972037 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:02.971884 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2555bb\"/\"openshift-service-ca.crt\"" Jul 13 19:41:02.980076 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:02.980043 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw"] Jul 13 19:41:03.043451 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.043415 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:41:03.043967 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.043937 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-home\") pod \"sub-test-model-2555bb-kserve-6d6469d9df-l6vqw\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.044088 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.044006 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-tmp-dir\") pod \"sub-test-model-2555bb-kserve-6d6469d9df-l6vqw\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.044088 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.044082 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-tls-certs\") pod \"sub-test-model-2555bb-kserve-6d6469d9df-l6vqw\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.044220 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.044199 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smgtp\" (UniqueName: \"kubernetes.io/projected/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-kube-api-access-smgtp\") pod \"sub-test-model-2555bb-kserve-6d6469d9df-l6vqw\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.044281 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.044250 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-dshm\") pod \"sub-test-model-2555bb-kserve-6d6469d9df-l6vqw\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.044324 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.044301 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-model-cache\") pod \"sub-test-model-2555bb-kserve-6d6469d9df-l6vqw\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.145278 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.145246 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-tmp-dir\") pod \"sub-test-model-2555bb-kserve-6d6469d9df-l6vqw\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.145491 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.145325 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-tls-certs\") pod \"sub-test-model-2555bb-kserve-6d6469d9df-l6vqw\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.145491 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.145363 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smgtp\" (UniqueName: \"kubernetes.io/projected/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-kube-api-access-smgtp\") pod \"sub-test-model-2555bb-kserve-6d6469d9df-l6vqw\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.145491 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.145397 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-dshm\") pod \"sub-test-model-2555bb-kserve-6d6469d9df-l6vqw\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.145701 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.145548 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-model-cache\") pod \"sub-test-model-2555bb-kserve-6d6469d9df-l6vqw\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.145701 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.145620 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-home\") pod \"sub-test-model-2555bb-kserve-6d6469d9df-l6vqw\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.145871 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.145846 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-tmp-dir\") pod \"sub-test-model-2555bb-kserve-6d6469d9df-l6vqw\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.146053 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.146024 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-model-cache\") pod \"sub-test-model-2555bb-kserve-6d6469d9df-l6vqw\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.146053 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.145999 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-home\") pod \"sub-test-model-2555bb-kserve-6d6469d9df-l6vqw\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.147877 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.147853 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-dshm\") pod \"sub-test-model-2555bb-kserve-6d6469d9df-l6vqw\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.148079 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.148065 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-tls-certs\") pod \"sub-test-model-2555bb-kserve-6d6469d9df-l6vqw\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.157108 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.157087 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smgtp\" (UniqueName: \"kubernetes.io/projected/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-kube-api-access-smgtp\") pod \"sub-test-model-2555bb-kserve-6d6469d9df-l6vqw\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.279166 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.279073 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:03.421507 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.421479 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw"] Jul 13 19:41:03.423795 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:41:03.423746 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf97bbc9e_9ae5_4cd6_b9cb_c1bb03079b13.slice/crio-14d6dc18a0fa4caaed1f75838ab04d4da82a92d6fb26eb47d8dff8c17c05c87e WatchSource:0}: Error finding container 14d6dc18a0fa4caaed1f75838ab04d4da82a92d6fb26eb47d8dff8c17c05c87e: Status 404 returned error can't find the container with id 14d6dc18a0fa4caaed1f75838ab04d4da82a92d6fb26eb47d8dff8c17c05c87e Jul 13 19:41:03.590078 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.590043 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" event={"ID":"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13","Type":"ContainerStarted","Data":"3aaaac6f8062823b299a4557209981fc106dbf85a0b55cd3e7c6543d0a621288"} Jul 13 19:41:03.590078 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.590085 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" event={"ID":"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13","Type":"ContainerStarted","Data":"14d6dc18a0fa4caaed1f75838ab04d4da82a92d6fb26eb47d8dff8c17c05c87e"} Jul 13 19:41:03.608082 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:03.608027 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" podStartSLOduration=1.608010728 podStartE2EDuration="1.608010728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:41:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:03.606583873 +0000 UTC m=+2549.083038064" watchObservedRunningTime="2026-07-13 19:41:03.608010728 +0000 UTC m=+2549.084464901" Jul 13 19:41:08.049648 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:08.049610 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:41:13.279828 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:13.279771 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:13.279828 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:13.279834 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:13.292016 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:13.291987 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:13.653716 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:13.653682 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:41:14.404191 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:14.404153 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:41:14.467417 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:14.467381 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:41:24.650382 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:24.650345 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:41:37.986075 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:37.986041 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:41:44.014301 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:44.014264 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:41:45.071184 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:45.071152 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw"] Jul 13 19:41:45.114013 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:45.113969 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:41:47.045435 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:41:47.045402 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:42:00.090836 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.090732 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" podUID="f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13" containerName="main" containerID="cri-o://3aaaac6f8062823b299a4557209981fc106dbf85a0b55cd3e7c6543d0a621288" gracePeriod=45 Jul 13 19:42:00.343305 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.343236 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:42:00.493217 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.493184 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-tls-certs\") pod \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " Jul 13 19:42:00.493381 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.493245 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smgtp\" (UniqueName: \"kubernetes.io/projected/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-kube-api-access-smgtp\") pod \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " Jul 13 19:42:00.493381 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.493274 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-tmp-dir\") pod \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " Jul 13 19:42:00.493381 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.493335 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-dshm\") pod \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " Jul 13 19:42:00.493381 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.493357 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-home\") pod \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " Jul 13 19:42:00.493541 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.493391 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-model-cache\") pod \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\" (UID: \"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13\") " Jul 13 19:42:00.493658 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.493588 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13" (UID: "f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:42:00.493821 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.493651 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-home" (OuterVolumeSpecName: "home") pod "f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13" (UID: "f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:42:00.493821 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.493731 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-model-cache" (OuterVolumeSpecName: "model-cache") pod "f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13" (UID: "f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:42:00.495431 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.495404 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-dshm" (OuterVolumeSpecName: "dshm") pod "f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13" (UID: "f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:42:00.495539 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.495460 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13" (UID: "f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:42:00.495539 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.495478 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-kube-api-access-smgtp" (OuterVolumeSpecName: "kube-api-access-smgtp") pod "f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13" (UID: "f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13"). InnerVolumeSpecName "kube-api-access-smgtp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:42:00.594351 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.594241 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:42:00.594351 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.594279 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:42:00.594351 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.594293 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:42:00.594351 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.594307 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:42:00.594351 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.594320 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smgtp\" (UniqueName: \"kubernetes.io/projected/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-kube-api-access-smgtp\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:42:00.594351 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.594332 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:42:00.791764 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.791726 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:42:00.887292 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.887191 2574 generic.go:356] "Generic (PLEG): container finished" podID="f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13" containerID="3aaaac6f8062823b299a4557209981fc106dbf85a0b55cd3e7c6543d0a621288" exitCode=0 Jul 13 19:42:00.887292 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.887236 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" Jul 13 19:42:00.887292 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.887265 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" event={"ID":"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13","Type":"ContainerDied","Data":"3aaaac6f8062823b299a4557209981fc106dbf85a0b55cd3e7c6543d0a621288"} Jul 13 19:42:00.887549 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.887312 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw" event={"ID":"f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13","Type":"ContainerDied","Data":"14d6dc18a0fa4caaed1f75838ab04d4da82a92d6fb26eb47d8dff8c17c05c87e"} Jul 13 19:42:00.887549 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.887335 2574 scope.go:117] "RemoveContainer" containerID="3aaaac6f8062823b299a4557209981fc106dbf85a0b55cd3e7c6543d0a621288" Jul 13 19:42:00.889715 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.889667 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:42:00.897266 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.897250 2574 scope.go:117] "RemoveContainer" containerID="3aaaac6f8062823b299a4557209981fc106dbf85a0b55cd3e7c6543d0a621288" Jul 13 19:42:00.897517 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:42:00.897498 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3aaaac6f8062823b299a4557209981fc106dbf85a0b55cd3e7c6543d0a621288\": container with ID starting with 3aaaac6f8062823b299a4557209981fc106dbf85a0b55cd3e7c6543d0a621288 not found: ID does not exist" containerID="3aaaac6f8062823b299a4557209981fc106dbf85a0b55cd3e7c6543d0a621288" Jul 13 19:42:00.897586 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.897528 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3aaaac6f8062823b299a4557209981fc106dbf85a0b55cd3e7c6543d0a621288"} err="failed to get container status \"3aaaac6f8062823b299a4557209981fc106dbf85a0b55cd3e7c6543d0a621288\": rpc error: code = NotFound desc = could not find container \"3aaaac6f8062823b299a4557209981fc106dbf85a0b55cd3e7c6543d0a621288\": container with ID starting with 3aaaac6f8062823b299a4557209981fc106dbf85a0b55cd3e7c6543d0a621288 not found: ID does not exist" Jul 13 19:42:00.909639 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.909616 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw"] Jul 13 19:42:00.917592 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:00.917572 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2555bb/sub-test-model-2555bb-kserve-6d6469d9df-l6vqw"] Jul 13 19:42:01.055772 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:01.055738 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13" path="/var/lib/kubelet/pods/f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13/volumes" Jul 13 19:42:04.740929 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:04.740891 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:42:10.928743 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:10.928701 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:42:13.133407 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.133370 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46"] Jul 13 19:42:13.133889 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.133872 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13" containerName="main" Jul 13 19:42:13.133889 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.133891 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13" containerName="main" Jul 13 19:42:13.133987 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.133969 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f97bbc9e-9ae5-4cd6-b9cb-c1bb03079b13" containerName="main" Jul 13 19:42:13.136986 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.136967 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.140515 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.140497 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-196ed1\"/\"default-dockercfg-x8ddf\"" Jul 13 19:42:13.141412 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.141392 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-196ed1\"/\"rate-test-model-196ed1-kserve-self-signed-certs\"" Jul 13 19:42:13.141636 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.141620 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-196ed1\"/\"openshift-service-ca.crt\"" Jul 13 19:42:13.141752 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.141722 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-196ed1\"/\"kube-root-ca.crt\"" Jul 13 19:42:13.151746 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.151722 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46"] Jul 13 19:42:13.174035 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.174002 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:42:13.218281 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.218250 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8z7p\" (UniqueName: \"kubernetes.io/projected/52d3753d-7259-488d-b535-9a1b0a7dd893-kube-api-access-g8z7p\") pod \"rate-test-model-196ed1-kserve-665ffd747f-4gf46\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.218430 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.218297 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-dshm\") pod \"rate-test-model-196ed1-kserve-665ffd747f-4gf46\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.218430 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.218317 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-home\") pod \"rate-test-model-196ed1-kserve-665ffd747f-4gf46\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.218430 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.218371 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52d3753d-7259-488d-b535-9a1b0a7dd893-tls-certs\") pod \"rate-test-model-196ed1-kserve-665ffd747f-4gf46\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.218430 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.218411 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-model-cache\") pod \"rate-test-model-196ed1-kserve-665ffd747f-4gf46\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.218568 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.218472 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-tmp-dir\") pod \"rate-test-model-196ed1-kserve-665ffd747f-4gf46\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.319569 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.319530 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-dshm\") pod \"rate-test-model-196ed1-kserve-665ffd747f-4gf46\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.319569 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.319573 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-home\") pod \"rate-test-model-196ed1-kserve-665ffd747f-4gf46\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.319884 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.319607 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52d3753d-7259-488d-b535-9a1b0a7dd893-tls-certs\") pod \"rate-test-model-196ed1-kserve-665ffd747f-4gf46\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.319884 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.319658 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-model-cache\") pod \"rate-test-model-196ed1-kserve-665ffd747f-4gf46\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.319884 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.319683 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-tmp-dir\") pod \"rate-test-model-196ed1-kserve-665ffd747f-4gf46\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.319884 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.319811 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8z7p\" (UniqueName: \"kubernetes.io/projected/52d3753d-7259-488d-b535-9a1b0a7dd893-kube-api-access-g8z7p\") pod \"rate-test-model-196ed1-kserve-665ffd747f-4gf46\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.320151 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.320125 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-home\") pod \"rate-test-model-196ed1-kserve-665ffd747f-4gf46\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.320211 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.320130 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-model-cache\") pod \"rate-test-model-196ed1-kserve-665ffd747f-4gf46\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.320211 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.320191 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-tmp-dir\") pod \"rate-test-model-196ed1-kserve-665ffd747f-4gf46\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.322027 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.322006 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-dshm\") pod \"rate-test-model-196ed1-kserve-665ffd747f-4gf46\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.322188 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.322171 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52d3753d-7259-488d-b535-9a1b0a7dd893-tls-certs\") pod \"rate-test-model-196ed1-kserve-665ffd747f-4gf46\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.328848 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.328820 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8z7p\" (UniqueName: \"kubernetes.io/projected/52d3753d-7259-488d-b535-9a1b0a7dd893-kube-api-access-g8z7p\") pod \"rate-test-model-196ed1-kserve-665ffd747f-4gf46\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.446859 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.446747 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:13.579010 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.578984 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46"] Jul 13 19:42:13.580363 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:42:13.580338 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52d3753d_7259_488d_b535_9a1b0a7dd893.slice/crio-dcc41c194d961c4b84ca5afd5b0c5febbcf5c17e3a5efe4cb44f8d354396be30 WatchSource:0}: Error finding container dcc41c194d961c4b84ca5afd5b0c5febbcf5c17e3a5efe4cb44f8d354396be30: Status 404 returned error can't find the container with id dcc41c194d961c4b84ca5afd5b0c5febbcf5c17e3a5efe4cb44f8d354396be30 Jul 13 19:42:13.960482 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.960451 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" event={"ID":"52d3753d-7259-488d-b535-9a1b0a7dd893","Type":"ContainerStarted","Data":"9c3b06e9f229bad1b661bb84302c40fe4229766d4554289f2785a0f587cf0783"} Jul 13 19:42:13.960482 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.960490 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" event={"ID":"52d3753d-7259-488d-b535-9a1b0a7dd893","Type":"ContainerStarted","Data":"dcc41c194d961c4b84ca5afd5b0c5febbcf5c17e3a5efe4cb44f8d354396be30"} Jul 13 19:42:13.980094 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:13.980045 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" podStartSLOduration=0.9800308 podStartE2EDuration="980.0308ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:42:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:42:13.97892154 +0000 UTC m=+2619.455375724" watchObservedRunningTime="2026-07-13 19:42:13.9800308 +0000 UTC m=+2619.456484974" Jul 13 19:42:16.950098 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:16.950059 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:42:23.447884 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:23.447844 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:23.448377 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:23.447901 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:23.460114 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:23.460083 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:24.022559 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:24.022530 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:42:31.680987 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:31.680942 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:42:31.781581 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:31.781541 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:42:38.753371 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.753338 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd"] Jul 13 19:42:38.758488 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.758467 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:38.761982 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.761957 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2555bb\"/\"openshift-service-ca.crt\"" Jul 13 19:42:38.762141 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.762009 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2555bb\"/\"kube-root-ca.crt\"" Jul 13 19:42:38.762141 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.762014 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2555bb\"/\"default-dockercfg-5cn5w\"" Jul 13 19:42:38.762141 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.762014 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2555bb\"/\"rate-test-model-2555bb-kserve-self-signed-certs\"" Jul 13 19:42:38.765571 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.765549 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd"] Jul 13 19:42:38.768679 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.768652 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-home\") pod \"rate-test-model-2555bb-kserve-b45b46887-knhpd\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:38.768810 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.768691 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-model-cache\") pod \"rate-test-model-2555bb-kserve-b45b46887-knhpd\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:38.768810 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.768744 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/888e0ba8-9b8f-4cce-9ac9-689531ebebce-tls-certs\") pod \"rate-test-model-2555bb-kserve-b45b46887-knhpd\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:38.768924 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.768835 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-dshm\") pod \"rate-test-model-2555bb-kserve-b45b46887-knhpd\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:38.768924 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.768870 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxjvs\" (UniqueName: \"kubernetes.io/projected/888e0ba8-9b8f-4cce-9ac9-689531ebebce-kube-api-access-sxjvs\") pod \"rate-test-model-2555bb-kserve-b45b46887-knhpd\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:38.768924 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.768894 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-tmp-dir\") pod \"rate-test-model-2555bb-kserve-b45b46887-knhpd\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:38.780894 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.780866 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:42:38.869414 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.869378 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxjvs\" (UniqueName: \"kubernetes.io/projected/888e0ba8-9b8f-4cce-9ac9-689531ebebce-kube-api-access-sxjvs\") pod \"rate-test-model-2555bb-kserve-b45b46887-knhpd\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:38.869414 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.869415 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-tmp-dir\") pod \"rate-test-model-2555bb-kserve-b45b46887-knhpd\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:38.869674 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.869543 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-home\") pod \"rate-test-model-2555bb-kserve-b45b46887-knhpd\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:38.869674 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.869575 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-model-cache\") pod \"rate-test-model-2555bb-kserve-b45b46887-knhpd\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:38.869674 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.869598 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/888e0ba8-9b8f-4cce-9ac9-689531ebebce-tls-certs\") pod \"rate-test-model-2555bb-kserve-b45b46887-knhpd\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:38.869674 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.869668 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-dshm\") pod \"rate-test-model-2555bb-kserve-b45b46887-knhpd\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:38.869892 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.869839 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-tmp-dir\") pod \"rate-test-model-2555bb-kserve-b45b46887-knhpd\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:38.869932 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.869906 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-home\") pod \"rate-test-model-2555bb-kserve-b45b46887-knhpd\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:38.869984 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.869965 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-model-cache\") pod \"rate-test-model-2555bb-kserve-b45b46887-knhpd\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:38.872005 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.871979 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-dshm\") pod \"rate-test-model-2555bb-kserve-b45b46887-knhpd\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:38.872270 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.872250 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/888e0ba8-9b8f-4cce-9ac9-689531ebebce-tls-certs\") pod \"rate-test-model-2555bb-kserve-b45b46887-knhpd\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:38.876862 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:38.876843 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxjvs\" (UniqueName: \"kubernetes.io/projected/888e0ba8-9b8f-4cce-9ac9-689531ebebce-kube-api-access-sxjvs\") pod \"rate-test-model-2555bb-kserve-b45b46887-knhpd\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:39.069694 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:39.069661 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:39.200942 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:39.200908 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd"] Jul 13 19:42:39.202445 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:42:39.202416 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod888e0ba8_9b8f_4cce_9ac9_689531ebebce.slice/crio-e16942d98c5e1e0a1f225de7bbb5f149b21053bee84f0ff6dc08c6e038a0879b WatchSource:0}: Error finding container e16942d98c5e1e0a1f225de7bbb5f149b21053bee84f0ff6dc08c6e038a0879b: Status 404 returned error can't find the container with id e16942d98c5e1e0a1f225de7bbb5f149b21053bee84f0ff6dc08c6e038a0879b Jul 13 19:42:39.899002 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:39.898971 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:42:40.000226 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:40.000188 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:42:40.095048 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:40.095012 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" event={"ID":"888e0ba8-9b8f-4cce-9ac9-689531ebebce","Type":"ContainerStarted","Data":"fd2eac77839433b5564d0586acd499719f4f6a37c61d018d693efe63518caf0c"} Jul 13 19:42:40.095048 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:40.095062 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" event={"ID":"888e0ba8-9b8f-4cce-9ac9-689531ebebce","Type":"ContainerStarted","Data":"e16942d98c5e1e0a1f225de7bbb5f149b21053bee84f0ff6dc08c6e038a0879b"} Jul 13 19:42:40.114491 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:40.114423 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" podStartSLOduration=2.114406324 podStartE2EDuration="2.114406324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:42:40.113406623 +0000 UTC m=+2645.589860798" watchObservedRunningTime="2026-07-13 19:42:40.114406324 +0000 UTC m=+2645.590860498" Jul 13 19:42:43.252064 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:43.252024 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:42:49.069909 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:49.069878 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:49.070511 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:49.069939 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:49.082325 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:49.082300 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:49.150693 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:49.150659 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:42:52.207042 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:52.207005 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:42:52.264326 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:42:52.264288 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:43:01.243979 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:01.243947 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:43:16.811134 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:16.811095 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:43:16.866947 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:16.866909 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:43:17.212330 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:17.212301 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:43:24.025593 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:24.025557 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46"] Jul 13 19:43:24.533372 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:24.533334 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:43:25.156234 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:25.156195 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd"] Jul 13 19:43:25.174021 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:25.173980 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:43:26.151039 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:26.150999 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7640-140178893340464"] Jul 13 19:43:26.155007 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:26.154990 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7640-140178893340464" Jul 13 19:43:26.157116 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:26.157094 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-wtckq\"" Jul 13 19:43:26.160409 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:26.160382 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7640-140178893340464"] Jul 13 19:43:26.232198 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:26.232157 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qnz6\" (UniqueName: \"kubernetes.io/projected/52086002-75c6-493e-a8b8-44df7415ef63-kube-api-access-9qnz6\") pod \"test-curl-7640-140178893340464\" (UID: \"52086002-75c6-493e-a8b8-44df7415ef63\") " pod="openshift-ingress/test-curl-7640-140178893340464" Jul 13 19:43:26.332940 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:26.332910 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qnz6\" (UniqueName: \"kubernetes.io/projected/52086002-75c6-493e-a8b8-44df7415ef63-kube-api-access-9qnz6\") pod \"test-curl-7640-140178893340464\" (UID: \"52086002-75c6-493e-a8b8-44df7415ef63\") " pod="openshift-ingress/test-curl-7640-140178893340464" Jul 13 19:43:26.340431 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:26.340403 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qnz6\" (UniqueName: \"kubernetes.io/projected/52086002-75c6-493e-a8b8-44df7415ef63-kube-api-access-9qnz6\") pod \"test-curl-7640-140178893340464\" (UID: \"52086002-75c6-493e-a8b8-44df7415ef63\") " pod="openshift-ingress/test-curl-7640-140178893340464" Jul 13 19:43:26.466648 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:26.466553 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7640-140178893340464" Jul 13 19:43:26.600117 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:26.600091 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7640-140178893340464"] Jul 13 19:43:26.601837 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:43:26.601806 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52086002_75c6_493e_a8b8_44df7415ef63.slice/crio-08c489ad8b3d54063634c84a7d36ca0e7ce407e12149347ec32ed8a481ba3905 WatchSource:0}: Error finding container 08c489ad8b3d54063634c84a7d36ca0e7ce407e12149347ec32ed8a481ba3905: Status 404 returned error can't find the container with id 08c489ad8b3d54063634c84a7d36ca0e7ce407e12149347ec32ed8a481ba3905 Jul 13 19:43:26.603179 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:26.603158 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 19:43:27.334694 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:27.334665 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7640-140178893340464" event={"ID":"52086002-75c6-493e-a8b8-44df7415ef63","Type":"ContainerStarted","Data":"08c489ad8b3d54063634c84a7d36ca0e7ce407e12149347ec32ed8a481ba3905"} Jul 13 19:43:28.341090 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:28.341057 2574 generic.go:356] "Generic (PLEG): container finished" podID="52086002-75c6-493e-a8b8-44df7415ef63" containerID="f07a74d2a055b241d12e82883857ec273150fdd277c5e4d9bcf425bfbdc9199d" exitCode=0 Jul 13 19:43:28.341498 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:28.341151 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7640-140178893340464" event={"ID":"52086002-75c6-493e-a8b8-44df7415ef63","Type":"ContainerDied","Data":"f07a74d2a055b241d12e82883857ec273150fdd277c5e4d9bcf425bfbdc9199d"} Jul 13 19:43:29.484375 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:29.484347 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7640-140178893340464" Jul 13 19:43:29.507967 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:29.507937 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7640-140178893340464_52086002-75c6-493e-a8b8-44df7415ef63/test-curl-7640-140178893340464/0.log" Jul 13 19:43:29.534592 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:29.534561 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7640-140178893340464"] Jul 13 19:43:29.536478 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:29.536459 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7640-140178893340464"] Jul 13 19:43:29.564559 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:29.564530 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qnz6\" (UniqueName: \"kubernetes.io/projected/52086002-75c6-493e-a8b8-44df7415ef63-kube-api-access-9qnz6\") pod \"52086002-75c6-493e-a8b8-44df7415ef63\" (UID: \"52086002-75c6-493e-a8b8-44df7415ef63\") " Jul 13 19:43:29.566741 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:29.566713 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52086002-75c6-493e-a8b8-44df7415ef63-kube-api-access-9qnz6" (OuterVolumeSpecName: "kube-api-access-9qnz6") pod "52086002-75c6-493e-a8b8-44df7415ef63" (UID: "52086002-75c6-493e-a8b8-44df7415ef63"). InnerVolumeSpecName "kube-api-access-9qnz6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:43:29.666049 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:29.665948 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qnz6\" (UniqueName: \"kubernetes.io/projected/52086002-75c6-493e-a8b8-44df7415ef63-kube-api-access-9qnz6\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:43:30.353354 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:30.353322 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08c489ad8b3d54063634c84a7d36ca0e7ce407e12149347ec32ed8a481ba3905" Jul 13 19:43:30.353354 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:30.353330 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7640-140178893340464" Jul 13 19:43:30.369373 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:43:30.369339 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7640-140178893340464\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="52086002-75c6-493e-a8b8-44df7415ef63" pod="openshift-ingress/test-curl-7640-140178893340464" Jul 13 19:43:31.054719 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:31.054682 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52086002-75c6-493e-a8b8-44df7415ef63" path="/var/lib/kubelet/pods/52086002-75c6-493e-a8b8-44df7415ef63/volumes" Jul 13 19:43:32.544401 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:32.544367 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:43:32.971393 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:32.971304 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7640-140178893795376"] Jul 13 19:43:32.971685 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:32.971671 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52086002-75c6-493e-a8b8-44df7415ef63" containerName="test-curl-7640-140178893340464" Jul 13 19:43:32.971729 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:32.971686 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="52086002-75c6-493e-a8b8-44df7415ef63" containerName="test-curl-7640-140178893340464" Jul 13 19:43:32.971798 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:32.971770 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="52086002-75c6-493e-a8b8-44df7415ef63" containerName="test-curl-7640-140178893340464" Jul 13 19:43:32.975095 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:32.975079 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7640-140178893795376" Jul 13 19:43:32.977912 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:32.977888 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-wtckq\"" Jul 13 19:43:32.979025 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:32.979001 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7640-140178893795376"] Jul 13 19:43:33.097518 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:33.097475 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqcqx\" (UniqueName: \"kubernetes.io/projected/e751e186-231d-4529-8cee-8b0d6d6d81b0-kube-api-access-tqcqx\") pod \"test-curl-7640-140178893795376\" (UID: \"e751e186-231d-4529-8cee-8b0d6d6d81b0\") " pod="openshift-ingress/test-curl-7640-140178893795376" Jul 13 19:43:33.198996 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:33.198946 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqcqx\" (UniqueName: \"kubernetes.io/projected/e751e186-231d-4529-8cee-8b0d6d6d81b0-kube-api-access-tqcqx\") pod \"test-curl-7640-140178893795376\" (UID: \"e751e186-231d-4529-8cee-8b0d6d6d81b0\") " pod="openshift-ingress/test-curl-7640-140178893795376" Jul 13 19:43:33.207443 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:33.207410 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqcqx\" (UniqueName: \"kubernetes.io/projected/e751e186-231d-4529-8cee-8b0d6d6d81b0-kube-api-access-tqcqx\") pod \"test-curl-7640-140178893795376\" (UID: \"e751e186-231d-4529-8cee-8b0d6d6d81b0\") " pod="openshift-ingress/test-curl-7640-140178893795376" Jul 13 19:43:33.287266 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:33.287183 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7640-140178893795376" Jul 13 19:43:33.418607 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:33.418582 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7640-140178893795376"] Jul 13 19:43:33.420593 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:43:33.420559 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode751e186_231d_4529_8cee_8b0d6d6d81b0.slice/crio-bb491eb1646188ad96b1935b22ed650fdd5a65823a866c286a32b48bf16257fe WatchSource:0}: Error finding container bb491eb1646188ad96b1935b22ed650fdd5a65823a866c286a32b48bf16257fe: Status 404 returned error can't find the container with id bb491eb1646188ad96b1935b22ed650fdd5a65823a866c286a32b48bf16257fe Jul 13 19:43:34.375147 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:34.375113 2574 generic.go:356] "Generic (PLEG): container finished" podID="e751e186-231d-4529-8cee-8b0d6d6d81b0" containerID="37a9ac4a1d1aca89cf19e152b96fe160465732198c44711362cd170bc8063011" exitCode=0 Jul 13 19:43:34.375530 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:34.375158 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7640-140178893795376" event={"ID":"e751e186-231d-4529-8cee-8b0d6d6d81b0","Type":"ContainerDied","Data":"37a9ac4a1d1aca89cf19e152b96fe160465732198c44711362cd170bc8063011"} Jul 13 19:43:34.375530 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:34.375205 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7640-140178893795376" event={"ID":"e751e186-231d-4529-8cee-8b0d6d6d81b0","Type":"ContainerStarted","Data":"bb491eb1646188ad96b1935b22ed650fdd5a65823a866c286a32b48bf16257fe"} Jul 13 19:43:35.172034 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:35.172003 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 19:43:35.179391 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:35.179368 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-79lb6_6f2dadfd-ce72-47b8-ac79-ca34bfbc27e7/ovn-acl-logging/0.log" Jul 13 19:43:35.522244 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:35.522213 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7640-140178893795376" Jul 13 19:43:35.542964 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:35.542934 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7640-140178893795376_e751e186-231d-4529-8cee-8b0d6d6d81b0/test-curl-7640-140178893795376/0.log" Jul 13 19:43:35.568060 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:35.568024 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7640-140178893795376"] Jul 13 19:43:35.571620 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:35.571593 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7640-140178893795376"] Jul 13 19:43:35.619894 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:35.619856 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqcqx\" (UniqueName: \"kubernetes.io/projected/e751e186-231d-4529-8cee-8b0d6d6d81b0-kube-api-access-tqcqx\") pod \"e751e186-231d-4529-8cee-8b0d6d6d81b0\" (UID: \"e751e186-231d-4529-8cee-8b0d6d6d81b0\") " Jul 13 19:43:35.622056 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:35.622027 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e751e186-231d-4529-8cee-8b0d6d6d81b0-kube-api-access-tqcqx" (OuterVolumeSpecName: "kube-api-access-tqcqx") pod "e751e186-231d-4529-8cee-8b0d6d6d81b0" (UID: "e751e186-231d-4529-8cee-8b0d6d6d81b0"). InnerVolumeSpecName "kube-api-access-tqcqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:43:35.697443 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:35.697357 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7640-140178893795376"] Jul 13 19:43:35.697802 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:35.697772 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e751e186-231d-4529-8cee-8b0d6d6d81b0" containerName="test-curl-7640-140178893795376" Jul 13 19:43:35.697802 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:35.697802 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e751e186-231d-4529-8cee-8b0d6d6d81b0" containerName="test-curl-7640-140178893795376" Jul 13 19:43:35.697962 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:35.697886 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e751e186-231d-4529-8cee-8b0d6d6d81b0" containerName="test-curl-7640-140178893795376" Jul 13 19:43:35.700958 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:35.700932 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7640-140178893795376" Jul 13 19:43:35.709479 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:35.709453 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7640-140178893795376"] Jul 13 19:43:35.721365 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:35.721340 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqcqx\" (UniqueName: \"kubernetes.io/projected/e751e186-231d-4529-8cee-8b0d6d6d81b0-kube-api-access-tqcqx\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:43:35.822654 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:35.822622 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mrhf\" (UniqueName: \"kubernetes.io/projected/2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3-kube-api-access-9mrhf\") pod \"test-curl-7640-140178893795376\" (UID: \"2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3\") " pod="openshift-ingress/test-curl-7640-140178893795376" Jul 13 19:43:35.923854 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:35.923824 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mrhf\" (UniqueName: \"kubernetes.io/projected/2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3-kube-api-access-9mrhf\") pod \"test-curl-7640-140178893795376\" (UID: \"2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3\") " pod="openshift-ingress/test-curl-7640-140178893795376" Jul 13 19:43:35.931886 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:35.931861 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mrhf\" (UniqueName: \"kubernetes.io/projected/2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3-kube-api-access-9mrhf\") pod \"test-curl-7640-140178893795376\" (UID: \"2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3\") " pod="openshift-ingress/test-curl-7640-140178893795376" Jul 13 19:43:36.010971 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:36.010869 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7640-140178893795376" Jul 13 19:43:36.144381 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:36.144358 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7640-140178893795376"] Jul 13 19:43:36.145826 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:43:36.145796 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2716ae13_f4e4_4dc1_9c7c_4ed437ec37f3.slice/crio-4ef66f0f469cea2eb0f39d560cae72eb59566d1f882f69b9eef23f2fca590184 WatchSource:0}: Error finding container 4ef66f0f469cea2eb0f39d560cae72eb59566d1f882f69b9eef23f2fca590184: Status 404 returned error can't find the container with id 4ef66f0f469cea2eb0f39d560cae72eb59566d1f882f69b9eef23f2fca590184 Jul 13 19:43:36.387631 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:36.387605 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7640-140178893795376" Jul 13 19:43:36.387631 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:36.387618 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb491eb1646188ad96b1935b22ed650fdd5a65823a866c286a32b48bf16257fe" Jul 13 19:43:36.388937 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:36.388902 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7640-140178893795376" event={"ID":"2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3","Type":"ContainerStarted","Data":"4ef66f0f469cea2eb0f39d560cae72eb59566d1f882f69b9eef23f2fca590184"} Jul 13 19:43:36.399492 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:36.399462 2574 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7640-140178893795376" oldPodUID="e751e186-231d-4529-8cee-8b0d6d6d81b0" podUID="2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3" Jul 13 19:43:37.055369 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:37.055331 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e751e186-231d-4529-8cee-8b0d6d6d81b0" path="/var/lib/kubelet/pods/e751e186-231d-4529-8cee-8b0d6d6d81b0/volumes" Jul 13 19:43:37.395674 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:37.395587 2574 generic.go:356] "Generic (PLEG): container finished" podID="2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3" containerID="f204b5696f46f3f4fd694d5ca53830be71bb0c92ffdc3b3038a1ac828d46b7dd" exitCode=0 Jul 13 19:43:37.395674 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:37.395657 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7640-140178893795376" event={"ID":"2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3","Type":"ContainerDied","Data":"f204b5696f46f3f4fd694d5ca53830be71bb0c92ffdc3b3038a1ac828d46b7dd"} Jul 13 19:43:38.538852 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:38.538826 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7640-140178893795376" Jul 13 19:43:38.560746 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:38.560719 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7640-140178893795376_2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3/test-curl-7640-140178893795376/0.log" Jul 13 19:43:38.589527 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:38.589494 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7640-140178893795376"] Jul 13 19:43:38.593698 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:38.593669 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7640-140178893795376"] Jul 13 19:43:38.652900 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:38.652816 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mrhf\" (UniqueName: \"kubernetes.io/projected/2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3-kube-api-access-9mrhf\") pod \"2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3\" (UID: \"2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3\") " Jul 13 19:43:38.654836 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:38.654801 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3-kube-api-access-9mrhf" (OuterVolumeSpecName: "kube-api-access-9mrhf") pod "2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3" (UID: "2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3"). InnerVolumeSpecName "kube-api-access-9mrhf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:43:38.753986 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:38.753949 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mrhf\" (UniqueName: \"kubernetes.io/projected/2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3-kube-api-access-9mrhf\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:43:39.046685 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.046641 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" podUID="52d3753d-7259-488d-b535-9a1b0a7dd893" containerName="main" containerID="cri-o://9c3b06e9f229bad1b661bb84302c40fe4229766d4554289f2785a0f587cf0783" gracePeriod=45 Jul 13 19:43:39.055378 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.055349 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3" path="/var/lib/kubelet/pods/2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3/volumes" Jul 13 19:43:39.299631 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.299566 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:43:39.406565 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.406531 2574 generic.go:356] "Generic (PLEG): container finished" podID="52d3753d-7259-488d-b535-9a1b0a7dd893" containerID="9c3b06e9f229bad1b661bb84302c40fe4229766d4554289f2785a0f587cf0783" exitCode=0 Jul 13 19:43:39.406739 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.406576 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" Jul 13 19:43:39.406739 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.406617 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" event={"ID":"52d3753d-7259-488d-b535-9a1b0a7dd893","Type":"ContainerDied","Data":"9c3b06e9f229bad1b661bb84302c40fe4229766d4554289f2785a0f587cf0783"} Jul 13 19:43:39.406739 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.406662 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46" event={"ID":"52d3753d-7259-488d-b535-9a1b0a7dd893","Type":"ContainerDied","Data":"dcc41c194d961c4b84ca5afd5b0c5febbcf5c17e3a5efe4cb44f8d354396be30"} Jul 13 19:43:39.406739 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.406687 2574 scope.go:117] "RemoveContainer" containerID="9c3b06e9f229bad1b661bb84302c40fe4229766d4554289f2785a0f587cf0783" Jul 13 19:43:39.407999 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.407827 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7640-140178893795376" Jul 13 19:43:39.415325 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.415305 2574 scope.go:117] "RemoveContainer" containerID="9c3b06e9f229bad1b661bb84302c40fe4229766d4554289f2785a0f587cf0783" Jul 13 19:43:39.415592 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:43:39.415575 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c3b06e9f229bad1b661bb84302c40fe4229766d4554289f2785a0f587cf0783\": container with ID starting with 9c3b06e9f229bad1b661bb84302c40fe4229766d4554289f2785a0f587cf0783 not found: ID does not exist" containerID="9c3b06e9f229bad1b661bb84302c40fe4229766d4554289f2785a0f587cf0783" Jul 13 19:43:39.415658 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.415599 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c3b06e9f229bad1b661bb84302c40fe4229766d4554289f2785a0f587cf0783"} err="failed to get container status \"9c3b06e9f229bad1b661bb84302c40fe4229766d4554289f2785a0f587cf0783\": rpc error: code = NotFound desc = could not find container \"9c3b06e9f229bad1b661bb84302c40fe4229766d4554289f2785a0f587cf0783\": container with ID starting with 9c3b06e9f229bad1b661bb84302c40fe4229766d4554289f2785a0f587cf0783 not found: ID does not exist" Jul 13 19:43:39.415658 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.415615 2574 scope.go:117] "RemoveContainer" containerID="f204b5696f46f3f4fd694d5ca53830be71bb0c92ffdc3b3038a1ac828d46b7dd" Jul 13 19:43:39.460880 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.460850 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52d3753d-7259-488d-b535-9a1b0a7dd893-tls-certs\") pod \"52d3753d-7259-488d-b535-9a1b0a7dd893\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " Jul 13 19:43:39.461033 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.460900 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8z7p\" (UniqueName: \"kubernetes.io/projected/52d3753d-7259-488d-b535-9a1b0a7dd893-kube-api-access-g8z7p\") pod \"52d3753d-7259-488d-b535-9a1b0a7dd893\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " Jul 13 19:43:39.461033 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.460938 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-dshm\") pod \"52d3753d-7259-488d-b535-9a1b0a7dd893\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " Jul 13 19:43:39.461033 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.460994 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-model-cache\") pod \"52d3753d-7259-488d-b535-9a1b0a7dd893\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " Jul 13 19:43:39.461033 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.461028 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-home\") pod \"52d3753d-7259-488d-b535-9a1b0a7dd893\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " Jul 13 19:43:39.461298 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.461125 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-tmp-dir\") pod \"52d3753d-7259-488d-b535-9a1b0a7dd893\" (UID: \"52d3753d-7259-488d-b535-9a1b0a7dd893\") " Jul 13 19:43:39.461358 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.461271 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-model-cache" (OuterVolumeSpecName: "model-cache") pod "52d3753d-7259-488d-b535-9a1b0a7dd893" (UID: "52d3753d-7259-488d-b535-9a1b0a7dd893"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:43:39.461421 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.461351 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-home" (OuterVolumeSpecName: "home") pod "52d3753d-7259-488d-b535-9a1b0a7dd893" (UID: "52d3753d-7259-488d-b535-9a1b0a7dd893"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:43:39.461488 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.461470 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "52d3753d-7259-488d-b535-9a1b0a7dd893" (UID: "52d3753d-7259-488d-b535-9a1b0a7dd893"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:43:39.461541 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.461526 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:43:39.461588 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.461550 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:43:39.463057 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.463039 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52d3753d-7259-488d-b535-9a1b0a7dd893-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "52d3753d-7259-488d-b535-9a1b0a7dd893" (UID: "52d3753d-7259-488d-b535-9a1b0a7dd893"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:43:39.463486 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.463470 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-dshm" (OuterVolumeSpecName: "dshm") pod "52d3753d-7259-488d-b535-9a1b0a7dd893" (UID: "52d3753d-7259-488d-b535-9a1b0a7dd893"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:43:39.463549 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.463501 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52d3753d-7259-488d-b535-9a1b0a7dd893-kube-api-access-g8z7p" (OuterVolumeSpecName: "kube-api-access-g8z7p") pod "52d3753d-7259-488d-b535-9a1b0a7dd893" (UID: "52d3753d-7259-488d-b535-9a1b0a7dd893"). InnerVolumeSpecName "kube-api-access-g8z7p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:43:39.562563 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.562450 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:43:39.562563 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.562498 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52d3753d-7259-488d-b535-9a1b0a7dd893-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:43:39.562563 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.562514 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8z7p\" (UniqueName: \"kubernetes.io/projected/52d3753d-7259-488d-b535-9a1b0a7dd893-kube-api-access-g8z7p\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:43:39.562563 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.562526 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52d3753d-7259-488d-b535-9a1b0a7dd893-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:43:39.727374 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.727337 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46"] Jul 13 19:43:39.731092 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:39.731066 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-196ed1/rate-test-model-196ed1-kserve-665ffd747f-4gf46"] Jul 13 19:43:40.176818 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.176763 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" podUID="888e0ba8-9b8f-4cce-9ac9-689531ebebce" containerName="main" containerID="cri-o://fd2eac77839433b5564d0586acd499719f4f6a37c61d018d693efe63518caf0c" gracePeriod=45 Jul 13 19:43:40.417312 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.417275 2574 generic.go:356] "Generic (PLEG): container finished" podID="888e0ba8-9b8f-4cce-9ac9-689531ebebce" containerID="fd2eac77839433b5564d0586acd499719f4f6a37c61d018d693efe63518caf0c" exitCode=0 Jul 13 19:43:40.417486 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.417351 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" event={"ID":"888e0ba8-9b8f-4cce-9ac9-689531ebebce","Type":"ContainerDied","Data":"fd2eac77839433b5564d0586acd499719f4f6a37c61d018d693efe63518caf0c"} Jul 13 19:43:40.434601 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.434543 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:43:40.572026 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.571987 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxjvs\" (UniqueName: \"kubernetes.io/projected/888e0ba8-9b8f-4cce-9ac9-689531ebebce-kube-api-access-sxjvs\") pod \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " Jul 13 19:43:40.572527 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.572077 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/888e0ba8-9b8f-4cce-9ac9-689531ebebce-tls-certs\") pod \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " Jul 13 19:43:40.572527 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.572118 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-home\") pod \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " Jul 13 19:43:40.572527 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.572134 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-tmp-dir\") pod \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " Jul 13 19:43:40.572527 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.572170 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-model-cache\") pod \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " Jul 13 19:43:40.572527 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.572196 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-dshm\") pod \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\" (UID: \"888e0ba8-9b8f-4cce-9ac9-689531ebebce\") " Jul 13 19:43:40.572527 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.572409 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-home" (OuterVolumeSpecName: "home") pod "888e0ba8-9b8f-4cce-9ac9-689531ebebce" (UID: "888e0ba8-9b8f-4cce-9ac9-689531ebebce"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:43:40.572527 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.572481 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "888e0ba8-9b8f-4cce-9ac9-689531ebebce" (UID: "888e0ba8-9b8f-4cce-9ac9-689531ebebce"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:43:40.572527 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.572499 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-model-cache" (OuterVolumeSpecName: "model-cache") pod "888e0ba8-9b8f-4cce-9ac9-689531ebebce" (UID: "888e0ba8-9b8f-4cce-9ac9-689531ebebce"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:43:40.574340 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.574314 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/888e0ba8-9b8f-4cce-9ac9-689531ebebce-kube-api-access-sxjvs" (OuterVolumeSpecName: "kube-api-access-sxjvs") pod "888e0ba8-9b8f-4cce-9ac9-689531ebebce" (UID: "888e0ba8-9b8f-4cce-9ac9-689531ebebce"). InnerVolumeSpecName "kube-api-access-sxjvs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:43:40.574340 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.574324 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-dshm" (OuterVolumeSpecName: "dshm") pod "888e0ba8-9b8f-4cce-9ac9-689531ebebce" (UID: "888e0ba8-9b8f-4cce-9ac9-689531ebebce"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:43:40.574501 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.574324 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/888e0ba8-9b8f-4cce-9ac9-689531ebebce-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "888e0ba8-9b8f-4cce-9ac9-689531ebebce" (UID: "888e0ba8-9b8f-4cce-9ac9-689531ebebce"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:43:40.673158 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.673118 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:43:40.673158 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.673152 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:43:40.673158 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.673161 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxjvs\" (UniqueName: \"kubernetes.io/projected/888e0ba8-9b8f-4cce-9ac9-689531ebebce-kube-api-access-sxjvs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:43:40.673399 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.673171 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/888e0ba8-9b8f-4cce-9ac9-689531ebebce-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:43:40.673399 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.673182 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:43:40.673399 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.673189 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/888e0ba8-9b8f-4cce-9ac9-689531ebebce-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:43:40.945885 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.945847 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7640-140178893796384"] Jul 13 19:43:40.946232 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.946220 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="888e0ba8-9b8f-4cce-9ac9-689531ebebce" containerName="main" Jul 13 19:43:40.946277 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.946234 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="888e0ba8-9b8f-4cce-9ac9-689531ebebce" containerName="main" Jul 13 19:43:40.946277 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.946253 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3" containerName="test-curl-7640-140178893795376" Jul 13 19:43:40.946277 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.946260 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3" containerName="test-curl-7640-140178893795376" Jul 13 19:43:40.946277 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.946268 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52d3753d-7259-488d-b535-9a1b0a7dd893" containerName="main" Jul 13 19:43:40.946277 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.946274 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="52d3753d-7259-488d-b535-9a1b0a7dd893" containerName="main" Jul 13 19:43:40.946426 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.946337 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="52d3753d-7259-488d-b535-9a1b0a7dd893" containerName="main" Jul 13 19:43:40.946426 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.946347 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2716ae13-f4e4-4dc1-9c7c-4ed437ec37f3" containerName="test-curl-7640-140178893795376" Jul 13 19:43:40.946426 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.946355 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="888e0ba8-9b8f-4cce-9ac9-689531ebebce" containerName="main" Jul 13 19:43:40.949486 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.949467 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7640-140178893796384" Jul 13 19:43:40.951718 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.951699 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-wtckq\"" Jul 13 19:43:40.955284 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:40.955250 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7640-140178893796384"] Jul 13 19:43:41.054607 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:41.054571 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52d3753d-7259-488d-b535-9a1b0a7dd893" path="/var/lib/kubelet/pods/52d3753d-7259-488d-b535-9a1b0a7dd893/volumes" Jul 13 19:43:41.076519 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:41.076494 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54xqh\" (UniqueName: \"kubernetes.io/projected/455834a3-c58c-45e4-a687-f4cb2f27a954-kube-api-access-54xqh\") pod \"test-curl-7640-140178893796384\" (UID: \"455834a3-c58c-45e4-a687-f4cb2f27a954\") " pod="openshift-ingress/test-curl-7640-140178893796384" Jul 13 19:43:41.177929 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:41.177889 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54xqh\" (UniqueName: \"kubernetes.io/projected/455834a3-c58c-45e4-a687-f4cb2f27a954-kube-api-access-54xqh\") pod \"test-curl-7640-140178893796384\" (UID: \"455834a3-c58c-45e4-a687-f4cb2f27a954\") " pod="openshift-ingress/test-curl-7640-140178893796384" Jul 13 19:43:41.186250 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:41.186221 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54xqh\" (UniqueName: \"kubernetes.io/projected/455834a3-c58c-45e4-a687-f4cb2f27a954-kube-api-access-54xqh\") pod \"test-curl-7640-140178893796384\" (UID: \"455834a3-c58c-45e4-a687-f4cb2f27a954\") " pod="openshift-ingress/test-curl-7640-140178893796384" Jul 13 19:43:41.260366 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:41.260271 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7640-140178893796384" Jul 13 19:43:41.404919 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:41.404896 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7640-140178893796384"] Jul 13 19:43:41.406675 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:43:41.406646 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod455834a3_c58c_45e4_a687_f4cb2f27a954.slice/crio-3dafe6dad06982cf08f6a36124fedf344d59646528d20923290ccd2226503733 WatchSource:0}: Error finding container 3dafe6dad06982cf08f6a36124fedf344d59646528d20923290ccd2226503733: Status 404 returned error can't find the container with id 3dafe6dad06982cf08f6a36124fedf344d59646528d20923290ccd2226503733 Jul 13 19:43:41.425484 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:41.425466 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" Jul 13 19:43:41.425564 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:41.425485 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd" event={"ID":"888e0ba8-9b8f-4cce-9ac9-689531ebebce","Type":"ContainerDied","Data":"e16942d98c5e1e0a1f225de7bbb5f149b21053bee84f0ff6dc08c6e038a0879b"} Jul 13 19:43:41.425564 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:41.425532 2574 scope.go:117] "RemoveContainer" containerID="fd2eac77839433b5564d0586acd499719f4f6a37c61d018d693efe63518caf0c" Jul 13 19:43:41.426994 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:41.426962 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7640-140178893796384" event={"ID":"455834a3-c58c-45e4-a687-f4cb2f27a954","Type":"ContainerStarted","Data":"3dafe6dad06982cf08f6a36124fedf344d59646528d20923290ccd2226503733"} Jul 13 19:43:41.441651 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:41.441628 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd"] Jul 13 19:43:41.446029 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:41.446005 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2555bb/rate-test-model-2555bb-kserve-b45b46887-knhpd"] Jul 13 19:43:42.433500 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:42.433465 2574 generic.go:356] "Generic (PLEG): container finished" podID="455834a3-c58c-45e4-a687-f4cb2f27a954" containerID="74e9cbaa72a4a027f73875657a76fcb80d107a425852a2b2babaeb7503d7cf80" exitCode=0 Jul 13 19:43:42.433907 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:42.433554 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7640-140178893796384" event={"ID":"455834a3-c58c-45e4-a687-f4cb2f27a954","Type":"ContainerDied","Data":"74e9cbaa72a4a027f73875657a76fcb80d107a425852a2b2babaeb7503d7cf80"} Jul 13 19:43:43.055073 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:43.055035 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="888e0ba8-9b8f-4cce-9ac9-689531ebebce" path="/var/lib/kubelet/pods/888e0ba8-9b8f-4cce-9ac9-689531ebebce/volumes" Jul 13 19:43:43.584445 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:43.584419 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7640-140178893796384" Jul 13 19:43:43.603550 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:43.603515 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7640-140178893796384_455834a3-c58c-45e4-a687-f4cb2f27a954/test-curl-7640-140178893796384/0.log" Jul 13 19:43:43.628737 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:43.628708 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7640-140178893796384"] Jul 13 19:43:43.632651 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:43.632625 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7640-140178893796384"] Jul 13 19:43:43.699317 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:43.699216 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54xqh\" (UniqueName: \"kubernetes.io/projected/455834a3-c58c-45e4-a687-f4cb2f27a954-kube-api-access-54xqh\") pod \"455834a3-c58c-45e4-a687-f4cb2f27a954\" (UID: \"455834a3-c58c-45e4-a687-f4cb2f27a954\") " Jul 13 19:43:43.701311 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:43.701283 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/455834a3-c58c-45e4-a687-f4cb2f27a954-kube-api-access-54xqh" (OuterVolumeSpecName: "kube-api-access-54xqh") pod "455834a3-c58c-45e4-a687-f4cb2f27a954" (UID: "455834a3-c58c-45e4-a687-f4cb2f27a954"). InnerVolumeSpecName "kube-api-access-54xqh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:43:43.799960 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:43.799923 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54xqh\" (UniqueName: \"kubernetes.io/projected/455834a3-c58c-45e4-a687-f4cb2f27a954-kube-api-access-54xqh\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:43:43.819903 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:43.819862 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196ed1-6c2h8"] Jul 13 19:43:43.820277 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:43.820262 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="455834a3-c58c-45e4-a687-f4cb2f27a954" containerName="test-curl-7640-140178893796384" Jul 13 19:43:43.820319 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:43.820280 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="455834a3-c58c-45e4-a687-f4cb2f27a954" containerName="test-curl-7640-140178893796384" Jul 13 19:43:43.820357 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:43.820345 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="455834a3-c58c-45e4-a687-f4cb2f27a954" containerName="test-curl-7640-140178893796384" Jul 13 19:43:43.823466 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:43.823450 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196ed1-6c2h8" Jul 13 19:43:43.825545 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:43.825522 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 19:43:43.825671 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:43.825583 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-llvfh\"" Jul 13 19:43:43.829938 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:43.829911 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196ed1-6c2h8"] Jul 13 19:43:43.900989 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:43.900953 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa6c3148-32eb-4a5d-831b-ed263de0bb82-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-196ed1-6c2h8\" (UID: \"aa6c3148-32eb-4a5d-831b-ed263de0bb82\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196ed1-6c2h8" Jul 13 19:43:44.002482 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:44.002389 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa6c3148-32eb-4a5d-831b-ed263de0bb82-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-196ed1-6c2h8\" (UID: \"aa6c3148-32eb-4a5d-831b-ed263de0bb82\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196ed1-6c2h8" Jul 13 19:43:44.003056 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:44.003037 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa6c3148-32eb-4a5d-831b-ed263de0bb82-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-196ed1-6c2h8\" (UID: \"aa6c3148-32eb-4a5d-831b-ed263de0bb82\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196ed1-6c2h8" Jul 13 19:43:44.135540 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:44.135508 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196ed1-6c2h8" Jul 13 19:43:44.269125 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:44.269102 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196ed1-6c2h8"] Jul 13 19:43:44.271057 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:43:44.271031 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa6c3148_32eb_4a5d_831b_ed263de0bb82.slice/crio-ad33512079600f5b67a838c0a9de3d58cbf0c9457db1a03c8c4fbe24f12da999 WatchSource:0}: Error finding container ad33512079600f5b67a838c0a9de3d58cbf0c9457db1a03c8c4fbe24f12da999: Status 404 returned error can't find the container with id ad33512079600f5b67a838c0a9de3d58cbf0c9457db1a03c8c4fbe24f12da999 Jul 13 19:43:44.334125 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:44.334085 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:43:44.431314 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:44.431282 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:43:44.446695 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:44.446671 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7640-140178893796384" Jul 13 19:43:44.446695 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:44.446690 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3dafe6dad06982cf08f6a36124fedf344d59646528d20923290ccd2226503733" Jul 13 19:43:44.447996 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:44.447969 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196ed1-6c2h8" event={"ID":"aa6c3148-32eb-4a5d-831b-ed263de0bb82","Type":"ContainerStarted","Data":"ad33512079600f5b67a838c0a9de3d58cbf0c9457db1a03c8c4fbe24f12da999"} Jul 13 19:43:44.456441 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:43:44.456414 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7640-140178893796384\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="455834a3-c58c-45e4-a687-f4cb2f27a954" pod="openshift-ingress/test-curl-7640-140178893796384" Jul 13 19:43:45.052594 ip-10-0-128-30 kubenswrapper[2574]: E0713 19:43:45.052558 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7640-140178893796384\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="455834a3-c58c-45e4-a687-f4cb2f27a954" pod="openshift-ingress/test-curl-7640-140178893796384" Jul 13 19:43:45.055174 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:45.055151 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="455834a3-c58c-45e4-a687-f4cb2f27a954" path="/var/lib/kubelet/pods/455834a3-c58c-45e4-a687-f4cb2f27a954/volumes" Jul 13 19:43:45.454370 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:45.454336 2574 generic.go:356] "Generic (PLEG): container finished" podID="aa6c3148-32eb-4a5d-831b-ed263de0bb82" containerID="ee6ce75844d129ae510a8260e366158e6c1f010b33477f6de58abd390c5563d2" exitCode=0 Jul 13 19:43:45.454544 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:45.454422 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196ed1-6c2h8" event={"ID":"aa6c3148-32eb-4a5d-831b-ed263de0bb82","Type":"ContainerDied","Data":"ee6ce75844d129ae510a8260e366158e6c1f010b33477f6de58abd390c5563d2"} Jul 13 19:43:46.603200 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:46.603177 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196ed1-6c2h8" Jul 13 19:43:46.732549 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:46.732456 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa6c3148-32eb-4a5d-831b-ed263de0bb82-maas-api-service-ca\") pod \"aa6c3148-32eb-4a5d-831b-ed263de0bb82\" (UID: \"aa6c3148-32eb-4a5d-831b-ed263de0bb82\") " Jul 13 19:43:46.732847 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:46.732822 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa6c3148-32eb-4a5d-831b-ed263de0bb82-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aa6c3148-32eb-4a5d-831b-ed263de0bb82" (UID: "aa6c3148-32eb-4a5d-831b-ed263de0bb82"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:43:46.834118 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:46.834078 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa6c3148-32eb-4a5d-831b-ed263de0bb82-maas-api-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:43:47.471095 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:47.471015 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196ed1-6c2h8" Jul 13 19:43:47.471095 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:47.471068 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196ed1-6c2h8" event={"ID":"aa6c3148-32eb-4a5d-831b-ed263de0bb82","Type":"ContainerDied","Data":"ad33512079600f5b67a838c0a9de3d58cbf0c9457db1a03c8c4fbe24f12da999"} Jul 13 19:43:47.471281 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:47.471107 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad33512079600f5b67a838c0a9de3d58cbf0c9457db1a03c8c4fbe24f12da999" Jul 13 19:43:54.301934 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:54.301902 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:43:57.937377 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:57.937338 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:43:58.917777 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:43:58.917741 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:44:04.693131 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:04.693093 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:44:04.802888 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:04.802854 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:44:09.593445 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:09.593404 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:44:09.830986 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:09.830946 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:44:09.938224 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:09.938147 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2555bb-zsc6h"] Jul 13 19:44:09.938572 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:09.938559 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa6c3148-32eb-4a5d-831b-ed263de0bb82" containerName="revoke-keys" Jul 13 19:44:09.938619 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:09.938573 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa6c3148-32eb-4a5d-831b-ed263de0bb82" containerName="revoke-keys" Jul 13 19:44:09.938654 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:09.938632 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa6c3148-32eb-4a5d-831b-ed263de0bb82" containerName="revoke-keys" Jul 13 19:44:09.943732 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:09.943710 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2555bb-zsc6h" Jul 13 19:44:09.946453 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:09.946428 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 19:44:09.946604 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:09.946499 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-llvfh\"" Jul 13 19:44:09.949871 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:09.949846 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2555bb-zsc6h"] Jul 13 19:44:10.042924 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:10.042886 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/345c4ea1-8a81-4d14-bfb7-afa7b2444212-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-2555bb-zsc6h\" (UID: \"345c4ea1-8a81-4d14-bfb7-afa7b2444212\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2555bb-zsc6h" Jul 13 19:44:10.144362 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:10.144326 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/345c4ea1-8a81-4d14-bfb7-afa7b2444212-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-2555bb-zsc6h\" (UID: \"345c4ea1-8a81-4d14-bfb7-afa7b2444212\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2555bb-zsc6h" Jul 13 19:44:10.145026 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:10.145004 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/345c4ea1-8a81-4d14-bfb7-afa7b2444212-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-2555bb-zsc6h\" (UID: \"345c4ea1-8a81-4d14-bfb7-afa7b2444212\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2555bb-zsc6h" Jul 13 19:44:10.256127 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:10.256043 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2555bb-zsc6h" Jul 13 19:44:10.390681 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:10.390653 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2555bb-zsc6h"] Jul 13 19:44:10.392376 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:44:10.392346 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod345c4ea1_8a81_4d14_bfb7_afa7b2444212.slice/crio-61b481948320ac5cb0095f312dd8ce24cfad0f2e5efb150aa343a68bb3b8c7f2 WatchSource:0}: Error finding container 61b481948320ac5cb0095f312dd8ce24cfad0f2e5efb150aa343a68bb3b8c7f2: Status 404 returned error can't find the container with id 61b481948320ac5cb0095f312dd8ce24cfad0f2e5efb150aa343a68bb3b8c7f2 Jul 13 19:44:10.461742 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:10.461706 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:44:10.576038 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:10.576003 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:44:10.589225 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:10.589197 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2555bb-zsc6h" event={"ID":"345c4ea1-8a81-4d14-bfb7-afa7b2444212","Type":"ContainerStarted","Data":"61b481948320ac5cb0095f312dd8ce24cfad0f2e5efb150aa343a68bb3b8c7f2"} Jul 13 19:44:11.595828 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:11.595774 2574 generic.go:356] "Generic (PLEG): container finished" podID="345c4ea1-8a81-4d14-bfb7-afa7b2444212" containerID="8cc14cf5e3658f8637a702ab87addcef4e398e66cbd138732754e86e6b3c2cdf" exitCode=0 Jul 13 19:44:11.596211 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:11.595866 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2555bb-zsc6h" event={"ID":"345c4ea1-8a81-4d14-bfb7-afa7b2444212","Type":"ContainerDied","Data":"8cc14cf5e3658f8637a702ab87addcef4e398e66cbd138732754e86e6b3c2cdf"} Jul 13 19:44:12.736303 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:12.736271 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2555bb-zsc6h" Jul 13 19:44:12.869530 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:12.869443 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/345c4ea1-8a81-4d14-bfb7-afa7b2444212-maas-api-service-ca\") pod \"345c4ea1-8a81-4d14-bfb7-afa7b2444212\" (UID: \"345c4ea1-8a81-4d14-bfb7-afa7b2444212\") " Jul 13 19:44:12.869852 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:12.869827 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/345c4ea1-8a81-4d14-bfb7-afa7b2444212-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "345c4ea1-8a81-4d14-bfb7-afa7b2444212" (UID: "345c4ea1-8a81-4d14-bfb7-afa7b2444212"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:44:12.970325 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:12.970279 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/345c4ea1-8a81-4d14-bfb7-afa7b2444212-maas-api-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\"" Jul 13 19:44:13.609711 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:13.609683 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2555bb-zsc6h" Jul 13 19:44:13.609910 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:13.609682 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2555bb-zsc6h" event={"ID":"345c4ea1-8a81-4d14-bfb7-afa7b2444212","Type":"ContainerDied","Data":"61b481948320ac5cb0095f312dd8ce24cfad0f2e5efb150aa343a68bb3b8c7f2"} Jul 13 19:44:13.609910 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:13.609820 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61b481948320ac5cb0095f312dd8ce24cfad0f2e5efb150aa343a68bb3b8c7f2" Jul 13 19:44:20.389821 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:20.389765 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:44:23.745206 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:23.745163 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k82x4"] Jul 13 19:44:25.033478 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:25.033429 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:44:31.156478 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:31.156441 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:44:31.226275 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:31.226235 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:44:35.755999 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:35.755960 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:44:35.988987 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:35.988952 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:44:41.116166 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:41.116126 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:44:41.935080 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:41.935051 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-559f5498f-l89sh_491181a6-8b08-403d-bb53-ddab58861a6a/copy-manifests/0.log" Jul 13 19:44:41.946961 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:41.943325 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-559f5498f-l89sh_491181a6-8b08-403d-bb53-ddab58861a6a/manager/0.log" Jul 13 19:44:43.155041 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:43.155013 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-868895-8nrlk_7582a728-a2cc-4f02-adac-8c0d3f0158f0/revoke-keys/0.log" Jul 13 19:44:43.269817 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:43.269778 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-f2d389-7rnjp_61dc57b6-95dc-4ce1-9102-11d1649db336/revoke-keys/0.log" Jul 13 19:44:43.612242 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:43.612209 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-0a1747-w7n5f_315a13c8-8080-443d-a63a-4b60636c5a42/revoke-keys/0.log" Jul 13 19:44:44.074743 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:44.074713 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-196ed1-6c2h8_aa6c3148-32eb-4a5d-831b-ed263de0bb82/revoke-keys/0.log" Jul 13 19:44:44.183147 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:44.183115 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-2555bb-zsc6h_345c4ea1-8a81-4d14-bfb7-afa7b2444212/revoke-keys/0.log" Jul 13 19:44:46.417105 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:46.417077 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-k82x4_e6559d43-6f70-440d-81fb-dbef1542cad1/limitador/0.log" Jul 13 19:44:46.537160 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:46.537123 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-574m7_1135c060-7bcd-4879-8212-4527f428e844/manager/0.log" Jul 13 19:44:47.117544 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:47.117495 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85c9d98b58-nzdqf_820904a9-84c2-4516-8669-5b979c4349f0/kube-auth-proxy/0.log" Jul 13 19:44:48.626503 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:48.626450 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-28fns_85de423e-4a1e-4cc4-a91c-c908f815e28e/main/0.log" Jul 13 19:44:52.546752 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:52.546710 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:44:52.624695 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:52.624631 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:44:55.647764 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:55.647736 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5ftlh_e8bde243-4f0c-4fd7-a18c-b5900c1609e8/global-pull-secret-syncer/0.log" Jul 13 19:44:55.870335 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:55.870296 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" Jul 13 19:44:56.013574 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:56.013490 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-30.ec2.internal_9f50767bdb9b137e0ddb50e76b305359/haproxy/0.log" Jul 13 19:44:57.059239 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:57.059161 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:44:57.119595 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:44:57.119557 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8v8bj_f079acce-c9cf-4ba0-b003-a176d76fa3ad/konnectivity-agent/0.log" limit=1024 Jul 13 19:45:00.138391 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:00.138358 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rfbbp"] Jul 13 19:45:00.138750 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:00.138737 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="345c4ea1-8a81-4d14-bfb7-afa7b2444212" containerName="revoke-keys" Jul 13 19:45:00.138750 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:00.138751 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="345c4ea1-8a81-4d14-bfb7-afa7b2444212" containerName="revoke-keys" Jul 13 19:45:00.138881 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:00.138837 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="345c4ea1-8a81-4d14-bfb7-afa7b2444212" containerName="revoke-keys" Jul 13 19:45:00.142125 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:00.142105 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rfbbp" Jul 13 19:45:00.144222 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:00.144203 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-llvfh\"" Jul 13 19:45:00.150066 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:00.149565 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rfbbp"] Jul 13 19:45:00.153232 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:00.153212 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rfbbp" Jul 13 19:45:00.287556 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:00.287529 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rfbbp"] Jul 13 19:45:00.290326 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:45:00.290300 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1720faa_cad4_42b9_9a73_a79f192fb97b.slice/crio-8b44fdbc1c236fa4932230262e039bfa91bd8749b8350b015abdb103a8add029 WatchSource:0}: Error finding container 8b44fdbc1c236fa4932230262e039bfa91bd8749b8350b015abdb103a8add029: Status 404 returned error can't find the container with id 8b44fdbc1c236fa4932230262e039bfa91bd8749b8350b015abdb103a8add029 Jul 13 19:45:00.858221 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:00.858185 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rfbbp" event={"ID":"c1720faa-cad4-42b9-9a73-a79f192fb97b","Type":"ContainerStarted","Data":"b3346190005217952bd6832732404af807fd02cf42c375fb97df0f3f0a20df0b"} Jul 13 19:45:00.858401 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:00.858239 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rfbbp" event={"ID":"c1720faa-cad4-42b9-9a73-a79f192fb97b","Type":"ContainerStarted","Data":"8b44fdbc1c236fa4932230262e039bfa91bd8749b8350b015abdb103a8add029"} Jul 13 19:45:00.875138 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:00.875089 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rfbbp" podStartSLOduration=0.875073497 podStartE2EDuration="875.073497ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:45:00.872613859 +0000 UTC m=+2786.349068054" watchObservedRunningTime="2026-07-13 19:45:00.875073497 +0000 UTC m=+2786.351527671" Jul 13 19:45:01.361300 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:01.361265 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-k82x4_e6559d43-6f70-440d-81fb-dbef1542cad1/limitador/0.log" Jul 13 19:45:01.472473 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:01.472433 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-574m7_1135c060-7bcd-4879-8212-4527f428e844/manager/0.log" Jul 13 19:45:01.864934 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:01.864903 2574 generic.go:356] "Generic (PLEG): container finished" podID="c1720faa-cad4-42b9-9a73-a79f192fb97b" containerID="b3346190005217952bd6832732404af807fd02cf42c375fb97df0f3f0a20df0b" exitCode=0 Jul 13 19:45:01.865122 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:01.864995 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rfbbp" event={"ID":"c1720faa-cad4-42b9-9a73-a79f192fb97b","Type":"ContainerDied","Data":"b3346190005217952bd6832732404af807fd02cf42c375fb97df0f3f0a20df0b"} Jul 13 19:45:03.004258 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.004231 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rfbbp" Jul 13 19:45:03.250631 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.250602 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f/alertmanager/0.log" Jul 13 19:45:03.275755 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.275711 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f/config-reloader/0.log" Jul 13 19:45:03.302941 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.302915 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f/kube-rbac-proxy-web/0.log" Jul 13 19:45:03.333525 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.333439 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f/kube-rbac-proxy/0.log" Jul 13 19:45:03.358205 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.358166 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f/kube-rbac-proxy-metric/0.log" Jul 13 19:45:03.383558 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.383526 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f/prom-label-proxy/0.log" Jul 13 19:45:03.407114 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.407085 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7b057d6f-c5a5-4d8b-b9fa-f1b0056a424f/init-config-reloader/0.log" Jul 13 19:45:03.477808 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.477742 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-99zld_3f3a9e5d-3247-474e-85fb-1026a2779ee1/kube-state-metrics/0.log" Jul 13 19:45:03.500943 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.500911 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-99zld_3f3a9e5d-3247-474e-85fb-1026a2779ee1/kube-rbac-proxy-main/0.log" Jul 13 19:45:03.532973 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.532934 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-99zld_3f3a9e5d-3247-474e-85fb-1026a2779ee1/kube-rbac-proxy-self/0.log" Jul 13 19:45:03.572955 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.572928 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-79664f77cb-c64s2_eaa1640a-5f83-4a1a-aa77-fbcbb321acb6/metrics-server/0.log" Jul 13 19:45:03.598593 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.598509 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-4m48s_eee86427-85f4-47aa-816e-754a5fd32f2d/monitoring-plugin/0.log" Jul 13 19:45:03.720686 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.720659 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8kqlr_4ca24210-be70-483b-9893-eece94d2756b/node-exporter/0.log" Jul 13 19:45:03.760840 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.760812 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8kqlr_4ca24210-be70-483b-9893-eece94d2756b/kube-rbac-proxy/0.log" Jul 13 19:45:03.790487 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.790461 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8kqlr_4ca24210-be70-483b-9893-eece94d2756b/init-textfile/0.log" Jul 13 19:45:03.875950 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.875928 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rfbbp" Jul 13 19:45:03.876126 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.875929 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rfbbp" event={"ID":"c1720faa-cad4-42b9-9a73-a79f192fb97b","Type":"ContainerDied","Data":"8b44fdbc1c236fa4932230262e039bfa91bd8749b8350b015abdb103a8add029"} Jul 13 19:45:03.876126 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.876083 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b44fdbc1c236fa4932230262e039bfa91bd8749b8350b015abdb103a8add029" Jul 13 19:45:03.905240 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.905203 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-b2htr_62c27f1a-06ed-42aa-a152-e96e9fc07478/kube-rbac-proxy-main/0.log" Jul 13 19:45:03.929712 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.929684 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-b2htr_62c27f1a-06ed-42aa-a152-e96e9fc07478/kube-rbac-proxy-self/0.log" Jul 13 19:45:03.958235 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:03.958202 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-b2htr_62c27f1a-06ed-42aa-a152-e96e9fc07478/openshift-state-metrics/0.log" Jul 13 19:45:04.007682 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.007652 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5e796353-dbba-44d4-a616-bd82bfd48cf3/prometheus/0.log" Jul 13 19:45:04.030582 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.030553 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5e796353-dbba-44d4-a616-bd82bfd48cf3/config-reloader/0.log" Jul 13 19:45:04.056077 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.056052 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5e796353-dbba-44d4-a616-bd82bfd48cf3/thanos-sidecar/0.log" Jul 13 19:45:04.077277 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.077251 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5e796353-dbba-44d4-a616-bd82bfd48cf3/kube-rbac-proxy-web/0.log" Jul 13 19:45:04.100735 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.100706 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5e796353-dbba-44d4-a616-bd82bfd48cf3/kube-rbac-proxy/0.log" Jul 13 19:45:04.125292 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.125156 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5e796353-dbba-44d4-a616-bd82bfd48cf3/kube-rbac-proxy-thanos/0.log" Jul 13 19:45:04.153115 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.153067 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5e796353-dbba-44d4-a616-bd82bfd48cf3/init-config-reloader/0.log" Jul 13 19:45:04.197992 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.197961 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-ckj74_9fc0bf7d-9486-4251-bc38-e671c0cf563b/prometheus-operator/0.log" Jul 13 19:45:04.229271 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.229234 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-ckj74_9fc0bf7d-9486-4251-bc38-e671c0cf563b/kube-rbac-proxy/0.log" Jul 13 19:45:04.256027 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.256003 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-bh8vz_8c4174fc-cb32-4f1b-bff6-8193e0d658a5/prometheus-operator-admission-webhook/0.log" Jul 13 19:45:04.292175 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.292148 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-cdcc49956-wnbmr_f739872a-0869-4559-9a2a-8690e8c630a6/telemeter-client/0.log" Jul 13 19:45:04.316105 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.316077 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-cdcc49956-wnbmr_f739872a-0869-4559-9a2a-8690e8c630a6/reload/0.log" Jul 13 19:45:04.338622 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.338591 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-cdcc49956-wnbmr_f739872a-0869-4559-9a2a-8690e8c630a6/kube-rbac-proxy/0.log" Jul 13 19:45:04.379364 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.379291 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b7c576458-z48dk_980c5a95-d03c-4018-9432-2db17d31c30b/thanos-query/0.log" Jul 13 19:45:04.400801 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.400762 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b7c576458-z48dk_980c5a95-d03c-4018-9432-2db17d31c30b/kube-rbac-proxy-web/0.log" Jul 13 19:45:04.424492 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.424464 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b7c576458-z48dk_980c5a95-d03c-4018-9432-2db17d31c30b/kube-rbac-proxy/0.log" Jul 13 19:45:04.448329 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.448300 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b7c576458-z48dk_980c5a95-d03c-4018-9432-2db17d31c30b/prom-label-proxy/0.log" Jul 13 19:45:04.473627 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.473604 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b7c576458-z48dk_980c5a95-d03c-4018-9432-2db17d31c30b/kube-rbac-proxy-rules/0.log" Jul 13 19:45:04.493948 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:04.493925 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b7c576458-z48dk_980c5a95-d03c-4018-9432-2db17d31c30b/kube-rbac-proxy-metrics/0.log" Jul 13 19:45:06.637066 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.637027 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9"] Jul 13 19:45:06.637625 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.637604 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1720faa-cad4-42b9-9a73-a79f192fb97b" containerName="cleanup" Jul 13 19:45:06.637711 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.637634 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1720faa-cad4-42b9-9a73-a79f192fb97b" containerName="cleanup" Jul 13 19:45:06.637804 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.637774 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1720faa-cad4-42b9-9a73-a79f192fb97b" containerName="cleanup" Jul 13 19:45:06.641121 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.641098 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:06.643247 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.643231 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hts9g\"/\"default-dockercfg-jhzhj\"" Jul 13 19:45:06.643422 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.643406 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hts9g\"/\"openshift-service-ca.crt\"" Jul 13 19:45:06.644022 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.644008 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hts9g\"/\"kube-root-ca.crt\"" Jul 13 19:45:06.650706 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.650682 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9"] Jul 13 19:45:06.775145 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.775104 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4960748-f9b1-4c41-8853-48f9618fa3c6-sys\") pod \"perf-node-gather-daemonset-zjbb9\" (UID: \"f4960748-f9b1-4c41-8853-48f9618fa3c6\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:06.775335 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.775172 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f4960748-f9b1-4c41-8853-48f9618fa3c6-proc\") pod \"perf-node-gather-daemonset-zjbb9\" (UID: \"f4960748-f9b1-4c41-8853-48f9618fa3c6\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:06.775335 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.775195 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6w8t\" (UniqueName: \"kubernetes.io/projected/f4960748-f9b1-4c41-8853-48f9618fa3c6-kube-api-access-j6w8t\") pod \"perf-node-gather-daemonset-zjbb9\" (UID: \"f4960748-f9b1-4c41-8853-48f9618fa3c6\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:06.775335 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.775225 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f4960748-f9b1-4c41-8853-48f9618fa3c6-podres\") pod \"perf-node-gather-daemonset-zjbb9\" (UID: \"f4960748-f9b1-4c41-8853-48f9618fa3c6\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:06.775335 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.775247 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4960748-f9b1-4c41-8853-48f9618fa3c6-lib-modules\") pod \"perf-node-gather-daemonset-zjbb9\" (UID: \"f4960748-f9b1-4c41-8853-48f9618fa3c6\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:06.876738 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.876691 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f4960748-f9b1-4c41-8853-48f9618fa3c6-proc\") pod \"perf-node-gather-daemonset-zjbb9\" (UID: \"f4960748-f9b1-4c41-8853-48f9618fa3c6\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:06.876738 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.876742 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6w8t\" (UniqueName: \"kubernetes.io/projected/f4960748-f9b1-4c41-8853-48f9618fa3c6-kube-api-access-j6w8t\") pod \"perf-node-gather-daemonset-zjbb9\" (UID: \"f4960748-f9b1-4c41-8853-48f9618fa3c6\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:06.876975 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.876841 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f4960748-f9b1-4c41-8853-48f9618fa3c6-proc\") pod \"perf-node-gather-daemonset-zjbb9\" (UID: \"f4960748-f9b1-4c41-8853-48f9618fa3c6\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:06.876975 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.876852 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f4960748-f9b1-4c41-8853-48f9618fa3c6-podres\") pod \"perf-node-gather-daemonset-zjbb9\" (UID: \"f4960748-f9b1-4c41-8853-48f9618fa3c6\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:06.876975 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.876885 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4960748-f9b1-4c41-8853-48f9618fa3c6-lib-modules\") pod \"perf-node-gather-daemonset-zjbb9\" (UID: \"f4960748-f9b1-4c41-8853-48f9618fa3c6\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:06.876975 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.876962 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4960748-f9b1-4c41-8853-48f9618fa3c6-sys\") pod \"perf-node-gather-daemonset-zjbb9\" (UID: \"f4960748-f9b1-4c41-8853-48f9618fa3c6\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:06.877139 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.877026 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f4960748-f9b1-4c41-8853-48f9618fa3c6-podres\") pod \"perf-node-gather-daemonset-zjbb9\" (UID: \"f4960748-f9b1-4c41-8853-48f9618fa3c6\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:06.877139 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.877067 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4960748-f9b1-4c41-8853-48f9618fa3c6-lib-modules\") pod \"perf-node-gather-daemonset-zjbb9\" (UID: \"f4960748-f9b1-4c41-8853-48f9618fa3c6\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:06.877139 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.877071 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4960748-f9b1-4c41-8853-48f9618fa3c6-sys\") pod \"perf-node-gather-daemonset-zjbb9\" (UID: \"f4960748-f9b1-4c41-8853-48f9618fa3c6\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:06.887245 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.887178 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6w8t\" (UniqueName: \"kubernetes.io/projected/f4960748-f9b1-4c41-8853-48f9618fa3c6-kube-api-access-j6w8t\") pod \"perf-node-gather-daemonset-zjbb9\" (UID: \"f4960748-f9b1-4c41-8853-48f9618fa3c6\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:06.952905 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.952869 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:06.961316 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:06.961289 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6bc895d6b8-xfbss_6018c5dd-8193-49fa-acf5-2311dd79a62b/console/0.log" Jul 13 19:45:07.018707 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:07.018660 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-xpkqs_3decbc78-37dd-41de-840a-eb02d2507656/download-server/0.log" Jul 13 19:45:07.091045 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:07.091020 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9"] Jul 13 19:45:07.093233 ip-10-0-128-30 kubenswrapper[2574]: W0713 19:45:07.093204 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf4960748_f9b1_4c41_8853_48f9618fa3c6.slice/crio-f5e9e3c31667d678754a93fd3f35d3846bdb545a8afcaceceda4b853eae06224 WatchSource:0}: Error finding container f5e9e3c31667d678754a93fd3f35d3846bdb545a8afcaceceda4b853eae06224: Status 404 returned error can't find the container with id f5e9e3c31667d678754a93fd3f35d3846bdb545a8afcaceceda4b853eae06224 Jul 13 19:45:07.896800 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:07.896751 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" event={"ID":"f4960748-f9b1-4c41-8853-48f9618fa3c6","Type":"ContainerStarted","Data":"6e4d08adfbd2583efebdcd8e25a87bc16ee766b013f70cbc4af7edb5321086c4"} Jul 13 19:45:07.897214 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:07.896818 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" event={"ID":"f4960748-f9b1-4c41-8853-48f9618fa3c6","Type":"ContainerStarted","Data":"f5e9e3c31667d678754a93fd3f35d3846bdb545a8afcaceceda4b853eae06224"} Jul 13 19:45:07.897214 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:07.896849 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:07.914138 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:07.914090 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" podStartSLOduration=1.9140735279999999 podStartE2EDuration="1.914073528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:45:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:45:07.9129077 +0000 UTC m=+2793.389361877" watchObservedRunningTime="2026-07-13 19:45:07.914073528 +0000 UTC m=+2793.390527702" Jul 13 19:45:08.813950 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:08.813921 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pvjsb_46876142-cde9-4280-924b-b72711f9af98/dns/0.log" Jul 13 19:45:08.855937 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:08.855906 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pvjsb_46876142-cde9-4280-924b-b72711f9af98/kube-rbac-proxy/0.log" Jul 13 19:45:09.015721 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:09.015694 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jczlv_32074cd4-753e-4257-9572-b2c21a540c81/dns-node-resolver/0.log" Jul 13 19:45:09.709598 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:09.709568 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xdbcs_1989f7e8-7848-4a80-b6f2-2344cdbe418e/node-ca/0.log" Jul 13 19:45:10.837410 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:10.837364 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85c9d98b58-nzdqf_820904a9-84c2-4516-8669-5b979c4349f0/kube-auth-proxy/0.log" Jul 13 19:45:11.811504 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:11.811465 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mdkrq_c8e76368-3984-43e3-a888-259acc7cd8d2/serve-healthcheck-canary/0.log" Jul 13 19:45:12.415517 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:12.415486 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wvtfn_86781ea3-12c5-4237-9a60-a0e514637a8a/kube-rbac-proxy/0.log" Jul 13 19:45:12.436645 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:12.436609 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wvtfn_86781ea3-12c5-4237-9a60-a0e514637a8a/exporter/0.log" Jul 13 19:45:12.458661 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:12.458628 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wvtfn_86781ea3-12c5-4237-9a60-a0e514637a8a/extractor/0.log" Jul 13 19:45:13.912539 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:13.912514 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-zjbb9" Jul 13 19:45:15.487753 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:15.487726 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-559f5498f-l89sh_491181a6-8b08-403d-bb53-ddab58861a6a/manager/0.log" Jul 13 19:45:15.522528 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:15.522504 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-559f5498f-l89sh_491181a6-8b08-403d-bb53-ddab58861a6a/copy-manifests/0.log" Jul 13 19:45:17.201700 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:17.201668 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-64bdc88f66-2dc2p_8a96a66d-188a-4dc6-abe7-cba90e7d793a/manager/0.log" Jul 13 19:45:17.307005 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:17.306977 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-qznmt_46c90dbb-e080-4d9d-bad5-bb0bd08c7b6b/openshift-lws-operator/0.log" Jul 13 19:45:18.036898 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:18.036863 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85c9d98b58-nzdqf_820904a9-84c2-4516-8669-5b979c4349f0/kube-auth-proxy/0.log" Jul 13 19:45:19.689675 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:19.689647 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85c9d98b58-nzdqf_820904a9-84c2-4516-8669-5b979c4349f0/kube-auth-proxy/0.log" Jul 13 19:45:24.164421 ip-10-0-128-30 kubenswrapper[2574]: I0713 19:45:24.164386 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4pbnd_1f5cf377-1e0d-49e3-ba52-dee61f029c7f/kube-multus/0.log"