Jul 09 01:57:31.276716 ip-10-0-130-184 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 09 01:57:31.276731 ip-10-0-130-184 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 09 01:57:31.276741 ip-10-0-130-184 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 09 01:57:31.277074 ip-10-0-130-184 systemd[1]: Failed to start Kubernetes Kubelet. Jul 09 01:57:41.525958 ip-10-0-130-184 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 09 01:57:41.525973 ip-10-0-130-184 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 11a44dfcbc974b27ad1c5fcb94fbbd57 -- Jul 09 01:59:39.183212 ip-10-0-130-184 systemd[1]: Starting Kubernetes Kubelet... Jul 09 01:59:39.582327 ip-10-0-130-184 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 01:59:39.582327 ip-10-0-130-184 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 01:59:39.582327 ip-10-0-130-184 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 01:59:39.582327 ip-10-0-130-184 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 01:59:39.582327 ip-10-0-130-184 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 01:59:39.585645 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.585518 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 01:59:39.592356 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592339 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 01:59:39.592356 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592353 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 01:59:39.592356 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592357 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592361 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592366 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592369 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592372 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592376 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592380 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592383 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592394 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592397 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592399 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592402 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592405 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592408 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592410 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592413 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592416 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592419 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592422 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592425 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 01:59:39.592450 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592428 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592431 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592434 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592436 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592439 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592442 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592445 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592447 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592450 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592453 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592456 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592459 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592462 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592465 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592468 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592471 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592474 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592477 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592480 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592483 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 01:59:39.593041 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592485 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592488 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592490 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592494 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592497 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592499 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592502 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592505 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592507 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592510 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592512 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592515 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592517 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592520 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592526 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592530 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592533 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592536 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592540 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 01:59:39.593552 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592544 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592548 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592551 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592554 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592558 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592562 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592565 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592568 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592570 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592573 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592577 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592579 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592582 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592585 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592587 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592591 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592595 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592598 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592601 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 01:59:39.594057 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592604 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592607 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592610 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592613 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592616 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592619 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592621 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592624 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592628 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592631 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592633 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592636 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592639 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592642 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592644 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592647 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592650 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592653 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.592656 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593391 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 01:59:39.594533 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593397 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593400 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593403 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593408 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593413 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593417 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593420 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593423 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593426 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593429 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593432 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593434 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593437 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593440 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593443 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593445 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593448 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593452 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593455 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593457 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 01:59:39.595046 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593462 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593466 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593468 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593471 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593474 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593477 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593480 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593483 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593486 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593489 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593491 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593494 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593497 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593499 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593502 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593505 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593507 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593511 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593513 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593516 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 01:59:39.595574 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593519 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593521 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593524 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593526 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593529 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593532 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593537 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593541 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593544 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593547 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593550 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593554 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593557 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593561 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593564 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593567 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593570 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593575 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593578 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 01:59:39.596172 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593580 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593583 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593585 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593588 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593591 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593593 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593596 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593599 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593602 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593605 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593607 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593610 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593613 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593616 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593618 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593621 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593624 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593627 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593629 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 01:59:39.596671 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593632 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593635 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593638 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593640 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593643 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593645 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593648 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593651 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593654 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593657 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593659 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593662 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593665 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593667 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593670 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593672 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593675 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593677 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593680 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.593683 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 01:59:39.597225 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594106 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594112 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594115 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594119 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594123 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594127 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594130 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594133 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594136 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594139 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594142 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594145 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594148 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594151 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594153 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594156 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594159 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594162 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594165 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594168 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 01:59:39.597732 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594170 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594173 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594176 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594179 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594183 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594185 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594188 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594191 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594194 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594197 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594199 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594202 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594204 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594207 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594210 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594213 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594215 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594218 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594221 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594224 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 01:59:39.598269 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594227 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594230 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594232 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594235 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594238 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594241 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594243 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594246 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594249 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594253 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594257 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594261 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594263 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594267 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594271 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594273 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594276 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594279 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594281 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594284 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 01:59:39.598812 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594287 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594290 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594292 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594295 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594297 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594300 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594303 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594305 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594308 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594310 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594316 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594319 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594322 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594324 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594327 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594329 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594332 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594335 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594337 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594340 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 01:59:39.599340 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594342 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594345 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594350 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594352 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594355 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594358 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594360 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594363 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594366 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594370 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594373 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594376 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594379 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594382 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594385 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594388 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594391 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594394 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.594396 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594460 2582 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 01:59:39.599854 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594467 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594473 2582 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594479 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594484 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594487 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594492 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594496 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594499 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594502 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594506 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594509 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594512 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594515 2582 flags.go:64] FLAG: --cgroup-root="" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594518 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594522 2582 flags.go:64] FLAG: --client-ca-file="" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594525 2582 flags.go:64] FLAG: --cloud-provider="external" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594528 2582 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594532 2582 flags.go:64] FLAG: --cluster-domain="" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594535 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594538 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594542 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594545 2582 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594549 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 01:59:39.600411 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594552 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594555 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594558 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594561 2582 flags.go:64] FLAG: --contention-profiling="false" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594564 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594567 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594571 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594573 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594578 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594581 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594583 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594587 2582 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594590 2582 flags.go:64] FLAG: --enable-server="true" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594593 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594598 2582 flags.go:64] FLAG: --event-burst="100" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594601 2582 flags.go:64] FLAG: --event-qps="50" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594604 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594607 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594610 2582 flags.go:64] FLAG: --eviction-hard="" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594614 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594617 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594620 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594623 2582 flags.go:64] FLAG: --eviction-soft="" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594626 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594629 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 01:59:39.601004 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594633 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594636 2582 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594639 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594642 2582 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594645 2582 flags.go:64] FLAG: --feature-gates="" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594648 2582 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594652 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594655 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594658 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594661 2582 flags.go:64] FLAG: --healthz-port="10248" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594664 2582 flags.go:64] FLAG: --help="false" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594667 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-130-184.ec2.internal" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594670 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594674 2582 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594677 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594680 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594684 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594687 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594690 2582 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594693 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594696 2582 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594700 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594704 2582 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594707 2582 flags.go:64] FLAG: --kube-reserved="" Jul 09 01:59:39.601688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594710 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594713 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594716 2582 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594719 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594722 2582 flags.go:64] FLAG: --lock-file="" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594725 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594732 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594735 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594741 2582 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594744 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594747 2582 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594750 2582 flags.go:64] FLAG: --logging-format="text" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594752 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594756 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594759 2582 flags.go:64] FLAG: --manifest-url="" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594763 2582 flags.go:64] FLAG: --manifest-url-header="" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594767 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594770 2582 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594774 2582 flags.go:64] FLAG: --max-pods="110" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594777 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594780 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594783 2582 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594786 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594789 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594792 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 01:59:39.602383 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594795 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594806 2582 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594809 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594812 2582 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594815 2582 flags.go:64] FLAG: --pod-cidr="" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594819 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594823 2582 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594826 2582 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594830 2582 flags.go:64] FLAG: --pods-per-core="0" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594833 2582 flags.go:64] FLAG: --port="10250" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594836 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594839 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-001bf4c8afa9f6d6b" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594842 2582 flags.go:64] FLAG: --qos-reserved="" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594845 2582 flags.go:64] FLAG: --read-only-port="10255" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594850 2582 flags.go:64] FLAG: --register-node="true" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594853 2582 flags.go:64] FLAG: --register-with-taints="" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594856 2582 flags.go:64] FLAG: --registry-burst="10" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594859 2582 flags.go:64] FLAG: --registry-qps="5" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594862 2582 flags.go:64] FLAG: --reserved-cpus="" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594865 2582 flags.go:64] FLAG: --reserved-memory="" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594869 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594872 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594875 2582 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594878 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594881 2582 flags.go:64] FLAG: --runonce="false" Jul 09 01:59:39.603034 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594884 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594888 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594891 2582 flags.go:64] FLAG: --seccomp-default="false" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594894 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594897 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594900 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594903 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594906 2582 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594909 2582 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594912 2582 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594915 2582 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594918 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594921 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594925 2582 flags.go:64] FLAG: --system-cgroups="" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594928 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594933 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594936 2582 flags.go:64] FLAG: --tls-cert-file="" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594939 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594956 2582 flags.go:64] FLAG: --tls-min-version="" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594960 2582 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594963 2582 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594967 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594971 2582 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594974 2582 flags.go:64] FLAG: --v="2" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594978 2582 flags.go:64] FLAG: --version="false" Jul 09 01:59:39.603664 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594982 2582 flags.go:64] FLAG: --vmodule="" Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594986 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.594989 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595120 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595126 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595129 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595132 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595136 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595139 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595142 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595144 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595147 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595150 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595153 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595156 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595160 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595164 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595167 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595169 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595172 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 01:59:39.604379 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595175 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595178 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595181 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595184 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595186 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595189 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595192 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595194 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595198 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595203 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595206 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595209 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595211 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595214 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595225 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595229 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595232 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595235 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595238 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595241 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 01:59:39.604914 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595244 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595246 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595249 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595252 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595255 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595258 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595260 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595265 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595269 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595273 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595275 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595279 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595282 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595285 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595289 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595292 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595295 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595298 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595300 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595303 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 01:59:39.605457 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595307 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595310 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595312 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595315 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595318 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595321 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595323 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595326 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595329 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595332 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595371 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595377 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595381 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595385 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595390 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595394 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595398 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595400 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595404 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595406 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 01:59:39.605974 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595409 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595412 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595416 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595418 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595421 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595424 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595427 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595430 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595434 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595436 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595439 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595442 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595445 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595448 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595451 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595454 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595456 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595459 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595462 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 01:59:39.606492 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595465 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 01:59:39.606983 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595468 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 01:59:39.606983 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.595471 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 01:59:39.606983 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.596097 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 01:59:39.606983 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.603021 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 09 01:59:39.606983 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.603042 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 01:59:39.606983 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.603922 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 01:59:39.606983 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.603929 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 01:59:39.606983 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.603994 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 09 01:59:39.606983 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604001 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 01:59:39.606983 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604004 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 01:59:39.606983 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604007 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 01:59:39.606983 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604010 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 01:59:39.606983 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604013 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 01:59:39.606983 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604016 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 01:59:39.606983 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604018 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604022 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604024 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604027 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604030 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604033 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604036 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604039 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604042 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604044 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604047 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604050 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604053 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604055 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604058 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604061 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604064 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604066 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604069 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604072 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 01:59:39.607398 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604075 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604078 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604083 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604085 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604088 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604091 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604096 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604099 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604102 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604105 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604108 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604110 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604114 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604118 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604121 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604124 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604127 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604129 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604132 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 01:59:39.607901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604135 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604137 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604140 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604143 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604146 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604149 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604153 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604158 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604160 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604163 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604166 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604169 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604172 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604175 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604178 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604181 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604184 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604187 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604190 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 01:59:39.608420 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604193 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604195 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604198 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604201 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604203 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604206 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604208 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604211 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604214 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604216 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604219 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604221 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604224 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604227 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604229 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604233 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604235 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604238 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604241 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604243 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 01:59:39.608901 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604246 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 01:59:39.609443 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604249 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 01:59:39.609443 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604251 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 01:59:39.609443 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604254 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 01:59:39.609443 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604256 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 01:59:39.609443 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604259 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 01:59:39.609443 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604262 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 01:59:39.609443 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604264 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 01:59:39.609443 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604267 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 01:59:39.609443 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604270 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 01:59:39.609443 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604273 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 01:59:39.609443 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604275 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 01:59:39.609443 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604278 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 01:59:39.609443 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604281 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 01:59:39.609443 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.604287 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 01:59:39.609443 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604402 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604407 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604410 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604413 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604415 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604418 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604421 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604423 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604427 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604429 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604432 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604435 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604437 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604440 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604443 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604445 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604448 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604450 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604453 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604455 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 01:59:39.609848 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604458 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604461 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604463 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604466 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604468 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604471 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604474 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604476 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604479 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604482 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604485 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604488 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604490 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604493 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604495 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604498 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604501 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604503 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604506 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 01:59:39.610378 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604509 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604511 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604514 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604516 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604519 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604522 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604524 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604527 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604530 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604532 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604535 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604538 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604542 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604545 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604548 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604550 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604553 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604556 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604559 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 01:59:39.610867 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604561 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604564 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604567 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604570 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604573 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604575 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604578 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604581 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604584 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604587 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604589 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604592 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604594 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604597 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604600 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604602 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604605 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604609 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604612 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 01:59:39.611594 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604616 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604618 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604621 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604624 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604627 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604630 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604633 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604636 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604639 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604641 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604644 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604647 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604650 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604652 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604655 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604658 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604660 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604663 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604666 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604669 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 01:59:39.612177 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604672 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 01:59:39.612696 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:39.604676 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 01:59:39.612696 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.604681 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 01:59:39.612696 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.604775 2582 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 01:59:39.612696 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.607094 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 01:59:39.612696 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.608461 2582 server.go:1019] "Starting client certificate rotation" Jul 09 01:59:39.612696 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.608561 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 01:59:39.612696 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.608626 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 01:59:39.612696 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.611178 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 01:59:39.617332 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.617312 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 01:59:39.631415 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.631394 2582 log.go:25] "Validated CRI v1 runtime API" Jul 09 01:59:39.633773 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.633760 2582 log.go:25] "Validated CRI v1 image API" Jul 09 01:59:39.634629 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.634612 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 01:59:39.637385 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.637359 2582 fs.go:135] Filesystem UUIDs: map[5da24fc3-1b2b-4d98-a6d1-97adca879cc7:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a9423705-c424-45cf-a928-4d5d5ac8aa76:/dev/nvme0n1p3] Jul 09 01:59:39.637444 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.637387 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 01:59:39.642821 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.642702 2582 manager.go:216] Machine: {Timestamp:2026-07-09 01:59:39.641145308 +0000 UTC m=+0.339199115 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3005161 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ef8237863c37c9e11ad828f0119ba SystemUUID:ec2ef823-7863-c37c-9e11-ad828f0119ba BootID:11a44dfc-bc97-4b27-ad1c-5fcb94fbbd57 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:69:d1:68:19:a1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:69:d1:68:19:a1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:56:77:61:a4:75:75 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 01:59:39.642821 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.642819 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 01:59:39.642956 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.642932 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 01:59:39.646544 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.646524 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 01:59:39.646682 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.646546 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-184.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 01:59:39.647497 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.647486 2582 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 01:59:39.647538 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.647500 2582 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 01:59:39.647538 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.647508 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 01:59:39.648135 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.648124 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 01:59:39.649391 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.649381 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 01:59:39.652487 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.652472 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 09 01:59:39.652604 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.652594 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 01:59:39.652646 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.652636 2582 kubelet.go:486] "Attempting to sync node with API server" Jul 09 01:59:39.652676 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.652664 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 01:59:39.652714 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.652684 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 01:59:39.652714 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.652698 2582 kubelet.go:398] "Adding apiserver pod source" Jul 09 01:59:39.652714 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.652711 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 01:59:39.653128 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.653116 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 01:59:39.653180 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.653137 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 01:59:39.653763 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.653732 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 01:59:39.655376 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.655358 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 09 01:59:39.657343 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.657328 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 01:59:39.657416 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.657360 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 01:59:39.658436 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.658424 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 01:59:39.658489 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.658459 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 01:59:39.658489 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.658466 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 01:59:39.658489 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.658488 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 01:59:39.658574 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.658499 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 01:59:39.658574 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.658505 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 01:59:39.658574 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.658513 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 01:59:39.658574 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.658518 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 01:59:39.658574 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.658525 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 01:59:39.658574 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.658530 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 01:59:39.658574 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.658546 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 01:59:39.658574 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.658551 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 01:59:39.659594 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.659582 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 01:59:39.659594 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.659593 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 01:59:39.663109 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.663087 2582 server.go:1268] "Started kubelet" Jul 09 01:59:39.663209 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.663141 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 01:59:39.663284 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.663266 2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-184.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 01:59:39.663422 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:39.663398 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-184.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 01:59:39.663422 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.663372 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 01:59:39.663531 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:39.663397 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 01:59:39.663531 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.663489 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 01:59:39.663932 ip-10-0-130-184 systemd[1]: Started Kubernetes Kubelet. Jul 09 01:59:39.667638 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.667613 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 01:59:39.669458 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:39.669433 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 01:59:39.669870 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.669857 2582 server.go:310] "Adding debug handlers to kubelet server" Jul 09 01:59:39.675329 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:39.674035 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-184.ec2.internal.18c07c88da8d971b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-184.ec2.internal,UID:ip-10-0-130-184.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-184.ec2.internal,},FirstTimestamp:2026-07-09 01:59:39.662825243 +0000 UTC m=+0.360879050,LastTimestamp:2026-07-09 01:59:39.662825243 +0000 UTC m=+0.360879050,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-184.ec2.internal,}" Jul 09 01:59:39.675616 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.675603 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 01:59:39.675661 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.675614 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 01:59:39.675721 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:39.675694 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-184.ec2.internal\" not found" Jul 09 01:59:39.676378 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.676361 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 01:59:39.676378 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.676365 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 01:59:39.676519 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.676387 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 01:59:39.676519 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.676489 2582 reconstruct.go:97] "Volume reconstruction finished" Jul 09 01:59:39.676519 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.676496 2582 reconciler.go:29] "Reconciler: start to sync state" Jul 09 01:59:39.676644 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.676577 2582 factory.go:153] Registering CRI-O factory Jul 09 01:59:39.676644 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.676591 2582 factory.go:223] Registration of the crio container factory successfully Jul 09 01:59:39.676716 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.676650 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 01:59:39.676716 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.676659 2582 factory.go:55] Registering systemd factory Jul 09 01:59:39.676716 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.676665 2582 factory.go:223] Registration of the systemd container factory successfully Jul 09 01:59:39.676716 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.676690 2582 factory.go:103] Registering Raw factory Jul 09 01:59:39.676716 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.676702 2582 manager.go:1196] Started watching for new ooms in manager Jul 09 01:59:39.677168 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.677155 2582 manager.go:349] Starting recovery of all containers Jul 09 01:59:39.686032 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:39.685996 2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-184.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 01:59:39.686443 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:39.686410 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 01:59:39.689491 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.689477 2582 manager.go:354] Recovery completed Jul 09 01:59:39.693697 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.693680 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 01:59:39.697813 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.697787 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 01:59:39.697892 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.697829 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 01:59:39.697892 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.697844 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 01:59:39.698367 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.698350 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 01:59:39.698437 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.698367 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 01:59:39.698437 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.698369 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 01:59:39.698437 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.698388 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 09 01:59:39.699816 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.699802 2582 policy_none.go:49] "None policy: Start" Jul 09 01:59:39.699885 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.699820 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 01:59:39.699885 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.699834 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 01:59:39.700041 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.700023 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 01:59:39.700082 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.700060 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 01:59:39.700082 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.700076 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 01:59:39.700140 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.700116 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sj5dw" Jul 09 01:59:39.700927 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.700916 2582 policy_none.go:47] "Start" Jul 09 01:59:39.711967 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.711840 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sj5dw" Jul 09 01:59:39.712532 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:39.712470 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-184.ec2.internal.18c07c88dca36c4e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-184.ec2.internal,UID:ip-10-0-130-184.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-184.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-184.ec2.internal,},FirstTimestamp:2026-07-09 01:59:39.69781051 +0000 UTC m=+0.395864321,LastTimestamp:2026-07-09 01:59:39.69781051 +0000 UTC m=+0.395864321,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-184.ec2.internal,}" Jul 09 01:59:39.737536 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.737517 2582 manager.go:337] "Starting Device Plugin manager" Jul 09 01:59:39.737598 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:39.737550 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 01:59:39.737598 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.737559 2582 server.go:87] "Starting device plugin registration server" Jul 09 01:59:39.737825 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.737813 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 01:59:39.737866 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.737827 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 01:59:39.737951 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.737927 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 01:59:39.738033 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.738018 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 01:59:39.738086 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.738035 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 01:59:39.738611 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:39.738586 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 01:59:39.738674 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:39.738636 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-184.ec2.internal\" not found" Jul 09 01:59:39.840037 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.839967 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 01:59:39.841275 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.841259 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 01:59:39.841337 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.841290 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 01:59:39.841337 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.841301 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 01:59:39.841337 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.841332 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-184.ec2.internal" Jul 09 01:59:39.853368 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.853349 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-184.ec2.internal" Jul 09 01:59:39.853426 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:39.853371 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-184.ec2.internal\": node \"ip-10-0-130-184.ec2.internal\" not found" Jul 09 01:59:39.857732 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.857706 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 01:59:39.859045 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.859026 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 01:59:39.859131 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.859062 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 01:59:39.859131 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.859083 2582 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 01:59:39.859131 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:39.859115 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 01:59:39.861963 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.861932 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 01:59:39.959733 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.959709 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-184.ec2.internal"] Jul 09 01:59:39.959819 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.959769 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 01:59:39.962263 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.962249 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 01:59:39.962322 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.962281 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 01:59:39.962322 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.962294 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 01:59:39.963543 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.963531 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 01:59:39.963681 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.963666 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal" Jul 09 01:59:39.963744 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.963709 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 01:59:39.964588 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.964572 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 01:59:39.964659 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.964595 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 01:59:39.964659 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.964618 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 01:59:39.964659 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.964629 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 01:59:39.964804 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.964602 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 01:59:39.964804 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.964693 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 01:59:39.966097 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.966080 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-184.ec2.internal" Jul 09 01:59:39.966181 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.966110 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 01:59:39.966851 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.966832 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 01:59:39.966943 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.966859 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 01:59:39.966943 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.966872 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 01:59:39.978414 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.978394 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fa270ef1e2da30182fb1ce8e87a3f7c2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal\" (UID: \"fa270ef1e2da30182fb1ce8e87a3f7c2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal" Jul 09 01:59:39.978498 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.978422 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa270ef1e2da30182fb1ce8e87a3f7c2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal\" (UID: \"fa270ef1e2da30182fb1ce8e87a3f7c2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal" Jul 09 01:59:39.978498 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:39.978440 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/81d8756a43625938ca8b36e75b20e52a-config\") pod \"kube-apiserver-proxy-ip-10-0-130-184.ec2.internal\" (UID: \"81d8756a43625938ca8b36e75b20e52a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-184.ec2.internal" Jul 09 01:59:39.990643 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:39.990623 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-184.ec2.internal\" not found" node="ip-10-0-130-184.ec2.internal" Jul 09 01:59:39.995248 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:39.995231 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-184.ec2.internal\" not found" node="ip-10-0-130-184.ec2.internal" Jul 09 01:59:40.037168 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:40.037146 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-184.ec2.internal\" not found" Jul 09 01:59:40.079630 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.079613 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fa270ef1e2da30182fb1ce8e87a3f7c2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal\" (UID: \"fa270ef1e2da30182fb1ce8e87a3f7c2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal" Jul 09 01:59:40.079723 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.079640 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa270ef1e2da30182fb1ce8e87a3f7c2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal\" (UID: \"fa270ef1e2da30182fb1ce8e87a3f7c2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal" Jul 09 01:59:40.079723 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.079658 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/81d8756a43625938ca8b36e75b20e52a-config\") pod \"kube-apiserver-proxy-ip-10-0-130-184.ec2.internal\" (UID: \"81d8756a43625938ca8b36e75b20e52a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-184.ec2.internal" Jul 09 01:59:40.079723 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.079704 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/81d8756a43625938ca8b36e75b20e52a-config\") pod \"kube-apiserver-proxy-ip-10-0-130-184.ec2.internal\" (UID: \"81d8756a43625938ca8b36e75b20e52a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-184.ec2.internal" Jul 09 01:59:40.079830 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.079719 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fa270ef1e2da30182fb1ce8e87a3f7c2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal\" (UID: \"fa270ef1e2da30182fb1ce8e87a3f7c2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal" Jul 09 01:59:40.079830 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.079731 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa270ef1e2da30182fb1ce8e87a3f7c2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal\" (UID: \"fa270ef1e2da30182fb1ce8e87a3f7c2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal" Jul 09 01:59:40.137460 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:40.137395 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-184.ec2.internal\" not found" Jul 09 01:59:40.238290 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:40.238267 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-184.ec2.internal\" not found" Jul 09 01:59:40.295294 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.295267 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal" Jul 09 01:59:40.298229 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.298208 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-184.ec2.internal" Jul 09 01:59:40.338352 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:40.338326 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-184.ec2.internal\" not found" Jul 09 01:59:40.442344 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:40.442239 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-184.ec2.internal\" not found" Jul 09 01:59:40.537167 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.537142 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 01:59:40.577534 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.577505 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal" Jul 09 01:59:40.601311 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.601291 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 01:59:40.603131 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.603116 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-184.ec2.internal" Jul 09 01:59:40.607897 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.607869 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 01:59:40.608021 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:40.608000 2582 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://aa9a1819ab1b844a1b10879a05e84552-f6e7448c748992fa.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.130.184:59616->100.50.75.73:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-184.ec2.internal" Jul 09 01:59:40.608021 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.608006 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 01:59:40.608110 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.608008 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 01:59:40.653394 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.653376 2582 apiserver.go:52] "Watching apiserver" Jul 09 01:59:40.676331 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.676313 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 01:59:40.685805 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.685789 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 01:59:40.687053 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.687034 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-k9d84","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm","openshift-image-registry/node-ca-qkgn2","openshift-multus/multus-additional-cni-plugins-6hktj","openshift-multus/network-metrics-daemon-trtrf","openshift-network-diagnostics/network-check-target-vgz69","openshift-ovn-kubernetes/ovnkube-node-lj28w","kube-system/global-pull-secret-syncer-lg95f","openshift-cluster-node-tuning-operator/tuned-w24jg","openshift-dns/node-resolver-bb6hq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal","openshift-multus/multus-mmnsx","openshift-network-operator/iptables-alerter-jbftz"] Jul 09 01:59:40.688658 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.688642 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.689910 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.689895 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.691297 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.691279 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qkgn2" Jul 09 01:59:40.694708 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.694076 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 01:59:40.694708 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.694149 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.694708 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.694181 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 01:59:40.694708 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.694527 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 01:59:40.695548 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.695443 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 01:59:40.695548 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.695460 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 01:59:40.695548 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.695509 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 01:59:40.695767 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.695511 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 01:59:40.695767 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.695567 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q4k4k\"" Jul 09 01:59:40.695767 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.695715 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:40.695992 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:40.695798 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trtrf" podUID="c5e07105-f26f-4ea9-8cde-9db06c456edc" Jul 09 01:59:40.696939 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.696924 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 01:59:40.697024 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:40.696987 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vgz69" podUID="1941492f-5a14-4095-8743-70cb9e39ebe1" Jul 09 01:59:40.698011 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.697997 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k9d84" Jul 09 01:59:40.698271 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.698252 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 01:59:40.698384 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.698367 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 01:59:40.699144 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.699127 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:40.699230 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:40.699187 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 01:59:40.700246 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.700215 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 01:59:40.700338 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.700253 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 01:59:40.700338 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.700256 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 01:59:40.700412 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.700224 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 01:59:40.700586 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.700568 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.700644 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.700599 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nrnww\"" Jul 09 01:59:40.700756 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.700739 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8zsvj\"" Jul 09 01:59:40.701207 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.700973 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fpc9q\"" Jul 09 01:59:40.701207 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.701021 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 01:59:40.701207 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.701050 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 01:59:40.701769 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.701754 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bb6hq" Jul 09 01:59:40.702595 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.702577 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 01:59:40.703161 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.703141 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.704087 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.704066 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 01:59:40.705058 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.704478 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jbftz" Jul 09 01:59:40.711503 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.711484 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 01:59:40.714649 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.714612 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 01:54:40 +0000 UTC" deadline="2027-12-07 02:27:57.973406254 +0000 UTC" Jul 09 01:59:40.714649 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.714648 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12384h28m17.258760466s" Jul 09 01:59:40.720674 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.720648 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 01:59:40.720825 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.720808 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 01:59:40.721047 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.721027 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 01:59:40.721147 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.721102 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 01:59:40.721378 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.721360 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 01:59:40.721495 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.721479 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 01:59:40.722575 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.722510 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 01:59:40.722575 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.722531 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zht4s\"" Jul 09 01:59:40.722751 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.722579 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-q6s9f\"" Jul 09 01:59:40.722751 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.722683 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-h8qrt\"" Jul 09 01:59:40.722751 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.722700 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xv9hb\"" Jul 09 01:59:40.722751 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.722683 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 01:59:40.723092 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.722866 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 01:59:40.723223 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.723206 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tvzl4\"" Jul 09 01:59:40.725519 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.725496 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 01:59:40.754239 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.754218 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tpscq" Jul 09 01:59:40.768225 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.768207 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tpscq" Jul 09 01:59:40.777881 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.777863 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 01:59:40.782688 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.782670 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9h9h\" (UniqueName: \"kubernetes.io/projected/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-kube-api-access-k9h9h\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.782747 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.782698 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db97ff5f-15c4-4cef-b16c-fa03e39df587-cni-binary-copy\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.782747 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.782717 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-modprobe-d\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.782747 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.782741 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qgtc\" (UniqueName: \"kubernetes.io/projected/5e4fedf3-163f-4a60-8177-eb75bd1950c3-kube-api-access-4qgtc\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.782860 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.782768 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-multus-cni-dir\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.782860 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.782797 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-host-run-multus-certs\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.782860 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.782826 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-sys\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.782860 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.782845 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-lib-modules\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.783020 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.782895 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret\") pod \"global-pull-secret-syncer-lg95f\" (UID: \"7d15267e-8d0e-49e2-9a69-9c85f84b645f\") " pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:40.783020 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.782918 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db97ff5f-15c4-4cef-b16c-fa03e39df587-system-cni-dir\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.783020 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.782954 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-multus-socket-dir-parent\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.783020 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.782970 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-slash\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.783020 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.782990 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-etc-openvswitch\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.783020 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783005 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1d33222e-bef7-4933-bfed-1a0ed60d9798-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.783237 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783022 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzggw\" (UniqueName: \"kubernetes.io/projected/db97ff5f-15c4-4cef-b16c-fa03e39df587-kube-api-access-tzggw\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.783237 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783037 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e4fedf3-163f-4a60-8177-eb75bd1950c3-tmp\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.783237 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783051 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-host-var-lib-cni-multus\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.783237 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783066 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-run-netns\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.783237 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783081 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-node-log\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.783237 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783110 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec9d7c46-2a09-420b-90ae-5aad678928b0-env-overrides\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.783237 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783140 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/83da97a5-3688-4c0d-97b6-cc3cc3ed3d25-host\") pod \"node-ca-qkgn2\" (UID: \"83da97a5-3688-4c0d-97b6-cc3cc3ed3d25\") " pod="openshift-image-registry/node-ca-qkgn2" Jul 09 01:59:40.783237 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783157 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a403528-9b59-4685-974e-69e52cac4ec4-hosts-file\") pod \"node-resolver-bb6hq\" (UID: \"2a403528-9b59-4685-974e-69e52cac4ec4\") " pod="openshift-dns/node-resolver-bb6hq" Jul 09 01:59:40.783237 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783174 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-multus-daemon-config\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.783237 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783193 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xrqm\" (UniqueName: \"kubernetes.io/projected/70bb0a63-1beb-4f24-bcb6-e2a95ac1f429-kube-api-access-2xrqm\") pod \"iptables-alerter-jbftz\" (UID: \"70bb0a63-1beb-4f24-bcb6-e2a95ac1f429\") " pod="openshift-network-operator/iptables-alerter-jbftz" Jul 09 01:59:40.783237 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783215 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-kubelet\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.783237 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783233 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-var-lib-openvswitch\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.783771 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783256 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-run-openvswitch\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.783771 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783269 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-sysconfig\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.783771 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783288 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/70bb0a63-1beb-4f24-bcb6-e2a95ac1f429-host-slash\") pod \"iptables-alerter-jbftz\" (UID: \"70bb0a63-1beb-4f24-bcb6-e2a95ac1f429\") " pod="openshift-network-operator/iptables-alerter-jbftz" Jul 09 01:59:40.783771 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783302 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ae3b2d7f-4db7-486c-b226-b37e4cfab1f8-konnectivity-ca\") pod \"konnectivity-agent-k9d84\" (UID: \"ae3b2d7f-4db7-486c-b226-b37e4cfab1f8\") " pod="kube-system/konnectivity-agent-k9d84" Jul 09 01:59:40.783771 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783334 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-multus-conf-dir\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.783771 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783377 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-log-socket\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.783771 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783414 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec9d7c46-2a09-420b-90ae-5aad678928b0-ovnkube-config\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.783771 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783454 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1d33222e-bef7-4933-bfed-1a0ed60d9798-socket-dir\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.783771 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783480 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-kubernetes\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.783771 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783508 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-cnibin\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.783771 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783531 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-cni-binary-copy\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.783771 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783558 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-host-var-lib-kubelet\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.783771 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783582 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-cni-netd\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.783771 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783606 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1d33222e-bef7-4933-bfed-1a0ed60d9798-device-dir\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.783771 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783624 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1d33222e-bef7-4933-bfed-1a0ed60d9798-etc-selinux\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.783771 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783647 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1d33222e-bef7-4933-bfed-1a0ed60d9798-sys-fs\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.784366 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783663 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-sysctl-conf\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.784366 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783699 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-os-release\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.784366 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783720 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-hostroot\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.784366 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783735 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-etc-kubernetes\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.784366 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783752 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec9d7c46-2a09-420b-90ae-5aad678928b0-ovn-node-metrics-cert\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.784366 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783767 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1d33222e-bef7-4933-bfed-1a0ed60d9798-registration-dir\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.784366 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783782 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db97ff5f-15c4-4cef-b16c-fa03e39df587-os-release\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.784366 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783795 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-systemd\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.784366 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783823 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ae3b2d7f-4db7-486c-b226-b37e4cfab1f8-agent-certs\") pod \"konnectivity-agent-k9d84\" (UID: \"ae3b2d7f-4db7-486c-b226-b37e4cfab1f8\") " pod="kube-system/konnectivity-agent-k9d84" Jul 09 01:59:40.784366 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783840 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-run-systemd\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.784366 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783866 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t64bp\" (UniqueName: \"kubernetes.io/projected/ec9d7c46-2a09-420b-90ae-5aad678928b0-kube-api-access-t64bp\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.784366 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783900 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db97ff5f-15c4-4cef-b16c-fa03e39df587-cnibin\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.784366 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783918 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/db97ff5f-15c4-4cef-b16c-fa03e39df587-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.784366 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783934 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9x76\" (UniqueName: \"kubernetes.io/projected/c5e07105-f26f-4ea9-8cde-9db06c456edc-kube-api-access-k9x76\") pod \"network-metrics-daemon-trtrf\" (UID: \"c5e07105-f26f-4ea9-8cde-9db06c456edc\") " pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:40.784366 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783953 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-var-lib-kubelet\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.784366 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783972 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2qkc\" (UniqueName: \"kubernetes.io/projected/2a403528-9b59-4685-974e-69e52cac4ec4-kube-api-access-s2qkc\") pod \"node-resolver-bb6hq\" (UID: \"2a403528-9b59-4685-974e-69e52cac4ec4\") " pod="openshift-dns/node-resolver-bb6hq" Jul 09 01:59:40.784863 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.783987 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtdsr\" (UniqueName: \"kubernetes.io/projected/83da97a5-3688-4c0d-97b6-cc3cc3ed3d25-kube-api-access-qtdsr\") pod \"node-ca-qkgn2\" (UID: \"83da97a5-3688-4c0d-97b6-cc3cc3ed3d25\") " pod="openshift-image-registry/node-ca-qkgn2" Jul 09 01:59:40.784863 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784003 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-host-run-k8s-cni-cncf-io\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.784863 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784018 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-run-ovn-kubernetes\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.784863 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784033 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnmsq\" (UniqueName: \"kubernetes.io/projected/1d33222e-bef7-4933-bfed-1a0ed60d9798-kube-api-access-pnmsq\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.784863 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784048 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-sysctl-d\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.784863 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784081 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7d15267e-8d0e-49e2-9a69-9c85f84b645f-kubelet-config\") pod \"global-pull-secret-syncer-lg95f\" (UID: \"7d15267e-8d0e-49e2-9a69-9c85f84b645f\") " pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:40.784863 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784105 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-host-run-netns\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.784863 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784122 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.784863 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784139 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/db97ff5f-15c4-4cef-b16c-fa03e39df587-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.784863 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784168 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-tuned\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.784863 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784181 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a403528-9b59-4685-974e-69e52cac4ec4-tmp-dir\") pod \"node-resolver-bb6hq\" (UID: \"2a403528-9b59-4685-974e-69e52cac4ec4\") " pod="openshift-dns/node-resolver-bb6hq" Jul 09 01:59:40.784863 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784199 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-system-cni-dir\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.784863 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784221 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs\") pod \"network-metrics-daemon-trtrf\" (UID: \"c5e07105-f26f-4ea9-8cde-9db06c456edc\") " pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:40.784863 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784251 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-run\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.784863 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784272 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-systemd-units\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.784863 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784286 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-run-ovn\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.785403 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784302 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-cni-bin\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.785403 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784317 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db97ff5f-15c4-4cef-b16c-fa03e39df587-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.785403 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784332 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/70bb0a63-1beb-4f24-bcb6-e2a95ac1f429-iptables-alerter-script\") pod \"iptables-alerter-jbftz\" (UID: \"70bb0a63-1beb-4f24-bcb6-e2a95ac1f429\") " pod="openshift-network-operator/iptables-alerter-jbftz" Jul 09 01:59:40.785403 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784359 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7d15267e-8d0e-49e2-9a69-9c85f84b645f-dbus\") pod \"global-pull-secret-syncer-lg95f\" (UID: \"7d15267e-8d0e-49e2-9a69-9c85f84b645f\") " pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:40.785403 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784383 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-host-var-lib-cni-bin\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.785403 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784409 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec9d7c46-2a09-420b-90ae-5aad678928b0-ovnkube-script-lib\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.785403 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784424 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmh2k\" (UniqueName: \"kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k\") pod \"network-check-target-vgz69\" (UID: \"1941492f-5a14-4095-8743-70cb9e39ebe1\") " pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 01:59:40.785403 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784448 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/83da97a5-3688-4c0d-97b6-cc3cc3ed3d25-serviceca\") pod \"node-ca-qkgn2\" (UID: \"83da97a5-3688-4c0d-97b6-cc3cc3ed3d25\") " pod="openshift-image-registry/node-ca-qkgn2" Jul 09 01:59:40.785403 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.784462 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-host\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.851426 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:40.851397 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81d8756a43625938ca8b36e75b20e52a.slice/crio-36e800c720176e8e4e7474309868c8e096ef0e05afe9f3688ddc019ab2a6e369 WatchSource:0}: Error finding container 36e800c720176e8e4e7474309868c8e096ef0e05afe9f3688ddc019ab2a6e369: Status 404 returned error can't find the container with id 36e800c720176e8e4e7474309868c8e096ef0e05afe9f3688ddc019ab2a6e369 Jul 09 01:59:40.851780 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:40.851756 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa270ef1e2da30182fb1ce8e87a3f7c2.slice/crio-0284ff7276563722aadcdee7cfa38937e46ba1fd1460f8e975e3557a35ca6e15 WatchSource:0}: Error finding container 0284ff7276563722aadcdee7cfa38937e46ba1fd1460f8e975e3557a35ca6e15: Status 404 returned error can't find the container with id 0284ff7276563722aadcdee7cfa38937e46ba1fd1460f8e975e3557a35ca6e15 Jul 09 01:59:40.856012 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.855996 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 01:59:40.861868 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.861830 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal" event={"ID":"fa270ef1e2da30182fb1ce8e87a3f7c2","Type":"ContainerStarted","Data":"0284ff7276563722aadcdee7cfa38937e46ba1fd1460f8e975e3557a35ca6e15"} Jul 09 01:59:40.863421 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.863399 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-184.ec2.internal" event={"ID":"81d8756a43625938ca8b36e75b20e52a","Type":"ContainerStarted","Data":"36e800c720176e8e4e7474309868c8e096ef0e05afe9f3688ddc019ab2a6e369"} Jul 09 01:59:40.885378 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885356 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-cnibin\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.885458 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885381 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-cni-binary-copy\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.885458 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885398 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-host-var-lib-kubelet\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.885458 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885414 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-cni-netd\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.885458 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885430 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1d33222e-bef7-4933-bfed-1a0ed60d9798-device-dir\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.885644 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885454 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1d33222e-bef7-4933-bfed-1a0ed60d9798-etc-selinux\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.885644 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885483 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1d33222e-bef7-4933-bfed-1a0ed60d9798-sys-fs\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.885644 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885498 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-host-var-lib-kubelet\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.885644 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885512 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-sysctl-conf\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.885644 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885504 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-cnibin\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.885644 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885540 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-os-release\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.885644 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885557 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1d33222e-bef7-4933-bfed-1a0ed60d9798-device-dir\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.885644 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885595 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-hostroot\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.885644 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885605 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-os-release\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.885644 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885622 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-etc-kubernetes\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.885644 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885628 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-cni-netd\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.885644 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885634 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1d33222e-bef7-4933-bfed-1a0ed60d9798-etc-selinux\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.885644 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885648 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-hostroot\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.886170 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885651 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec9d7c46-2a09-420b-90ae-5aad678928b0-ovn-node-metrics-cert\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.886170 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885677 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-etc-kubernetes\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.886170 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885690 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1d33222e-bef7-4933-bfed-1a0ed60d9798-registration-dir\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.886170 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885705 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-sysctl-conf\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.886170 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885719 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db97ff5f-15c4-4cef-b16c-fa03e39df587-os-release\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.886170 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885697 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1d33222e-bef7-4933-bfed-1a0ed60d9798-sys-fs\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.886170 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885748 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-systemd\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.886170 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885768 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1d33222e-bef7-4933-bfed-1a0ed60d9798-registration-dir\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.886170 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885781 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ae3b2d7f-4db7-486c-b226-b37e4cfab1f8-agent-certs\") pod \"konnectivity-agent-k9d84\" (UID: \"ae3b2d7f-4db7-486c-b226-b37e4cfab1f8\") " pod="kube-system/konnectivity-agent-k9d84" Jul 09 01:59:40.886170 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885802 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db97ff5f-15c4-4cef-b16c-fa03e39df587-os-release\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.886170 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885808 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-run-systemd\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.886170 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885825 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-systemd\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.886170 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885843 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t64bp\" (UniqueName: \"kubernetes.io/projected/ec9d7c46-2a09-420b-90ae-5aad678928b0-kube-api-access-t64bp\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.886170 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885849 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-run-systemd\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.886170 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885879 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db97ff5f-15c4-4cef-b16c-fa03e39df587-cnibin\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.886170 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885910 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/db97ff5f-15c4-4cef-b16c-fa03e39df587-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.886170 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885952 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db97ff5f-15c4-4cef-b16c-fa03e39df587-cnibin\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.887058 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.885962 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9x76\" (UniqueName: \"kubernetes.io/projected/c5e07105-f26f-4ea9-8cde-9db06c456edc-kube-api-access-k9x76\") pod \"network-metrics-daemon-trtrf\" (UID: \"c5e07105-f26f-4ea9-8cde-9db06c456edc\") " pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:40.887058 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886000 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 01:59:40.887058 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886056 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-var-lib-kubelet\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.887058 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886008 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-var-lib-kubelet\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.887058 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886101 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2qkc\" (UniqueName: \"kubernetes.io/projected/2a403528-9b59-4685-974e-69e52cac4ec4-kube-api-access-s2qkc\") pod \"node-resolver-bb6hq\" (UID: \"2a403528-9b59-4685-974e-69e52cac4ec4\") " pod="openshift-dns/node-resolver-bb6hq" Jul 09 01:59:40.887058 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886131 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-cni-binary-copy\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.887058 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886172 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtdsr\" (UniqueName: \"kubernetes.io/projected/83da97a5-3688-4c0d-97b6-cc3cc3ed3d25-kube-api-access-qtdsr\") pod \"node-ca-qkgn2\" (UID: \"83da97a5-3688-4c0d-97b6-cc3cc3ed3d25\") " pod="openshift-image-registry/node-ca-qkgn2" Jul 09 01:59:40.887058 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886203 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-host-run-k8s-cni-cncf-io\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.887058 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886272 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-host-run-k8s-cni-cncf-io\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.887058 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886267 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-run-ovn-kubernetes\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.887058 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886350 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnmsq\" (UniqueName: \"kubernetes.io/projected/1d33222e-bef7-4933-bfed-1a0ed60d9798-kube-api-access-pnmsq\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.887058 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886380 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-sysctl-d\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.887058 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886388 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-run-ovn-kubernetes\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.887058 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886408 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7d15267e-8d0e-49e2-9a69-9c85f84b645f-kubelet-config\") pod \"global-pull-secret-syncer-lg95f\" (UID: \"7d15267e-8d0e-49e2-9a69-9c85f84b645f\") " pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:40.887058 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886438 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-host-run-netns\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.887058 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886470 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.887058 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886544 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/db97ff5f-15c4-4cef-b16c-fa03e39df587-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.887864 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886605 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-tuned\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.887864 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886625 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-host-run-netns\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.887864 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886630 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a403528-9b59-4685-974e-69e52cac4ec4-tmp-dir\") pod \"node-resolver-bb6hq\" (UID: \"2a403528-9b59-4685-974e-69e52cac4ec4\") " pod="openshift-dns/node-resolver-bb6hq" Jul 09 01:59:40.887864 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886675 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-system-cni-dir\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.887864 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886676 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/db97ff5f-15c4-4cef-b16c-fa03e39df587-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.887864 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886715 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7d15267e-8d0e-49e2-9a69-9c85f84b645f-kubelet-config\") pod \"global-pull-secret-syncer-lg95f\" (UID: \"7d15267e-8d0e-49e2-9a69-9c85f84b645f\") " pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:40.887864 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886716 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs\") pod \"network-metrics-daemon-trtrf\" (UID: \"c5e07105-f26f-4ea9-8cde-9db06c456edc\") " pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:40.887864 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886759 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-system-cni-dir\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.887864 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886788 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-run\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.887864 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886834 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.887864 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:40.886865 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 01:59:40.887864 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886883 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-systemd-units\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.887864 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:40.886930 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs podName:c5e07105-f26f-4ea9-8cde-9db06c456edc nodeName:}" failed. No retries permitted until 2026-07-09 01:59:41.386900178 +0000 UTC m=+2.084953996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs") pod "network-metrics-daemon-trtrf" (UID: "c5e07105-f26f-4ea9-8cde-9db06c456edc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 01:59:40.887864 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886989 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-run-ovn\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.887864 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887001 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-run\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.887864 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887001 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-systemd-units\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.887864 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887021 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-cni-bin\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.888704 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887065 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db97ff5f-15c4-4cef-b16c-fa03e39df587-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.888704 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887101 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/70bb0a63-1beb-4f24-bcb6-e2a95ac1f429-iptables-alerter-script\") pod \"iptables-alerter-jbftz\" (UID: \"70bb0a63-1beb-4f24-bcb6-e2a95ac1f429\") " pod="openshift-network-operator/iptables-alerter-jbftz" Jul 09 01:59:40.888704 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887133 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7d15267e-8d0e-49e2-9a69-9c85f84b645f-dbus\") pod \"global-pull-secret-syncer-lg95f\" (UID: \"7d15267e-8d0e-49e2-9a69-9c85f84b645f\") " pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:40.888704 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887161 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-host-var-lib-cni-bin\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.888704 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887187 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec9d7c46-2a09-420b-90ae-5aad678928b0-ovnkube-script-lib\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.888704 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887193 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-run-ovn\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.888704 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887162 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-cni-bin\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.888704 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886924 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a403528-9b59-4685-974e-69e52cac4ec4-tmp-dir\") pod \"node-resolver-bb6hq\" (UID: \"2a403528-9b59-4685-974e-69e52cac4ec4\") " pod="openshift-dns/node-resolver-bb6hq" Jul 09 01:59:40.888704 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887224 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmh2k\" (UniqueName: \"kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k\") pod \"network-check-target-vgz69\" (UID: \"1941492f-5a14-4095-8743-70cb9e39ebe1\") " pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 01:59:40.888704 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887286 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/db97ff5f-15c4-4cef-b16c-fa03e39df587-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.888704 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887253 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/83da97a5-3688-4c0d-97b6-cc3cc3ed3d25-serviceca\") pod \"node-ca-qkgn2\" (UID: \"83da97a5-3688-4c0d-97b6-cc3cc3ed3d25\") " pod="openshift-image-registry/node-ca-qkgn2" Jul 09 01:59:40.888704 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887334 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7d15267e-8d0e-49e2-9a69-9c85f84b645f-dbus\") pod \"global-pull-secret-syncer-lg95f\" (UID: \"7d15267e-8d0e-49e2-9a69-9c85f84b645f\") " pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:40.888704 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.886574 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-sysctl-d\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.888704 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887394 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-host-var-lib-cni-bin\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.888704 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887409 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db97ff5f-15c4-4cef-b16c-fa03e39df587-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.888704 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887560 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-host\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.888704 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887607 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9h9h\" (UniqueName: \"kubernetes.io/projected/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-kube-api-access-k9h9h\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.889706 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887675 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db97ff5f-15c4-4cef-b16c-fa03e39df587-cni-binary-copy\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.889706 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887751 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-host\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.889706 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.887768 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-modprobe-d\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.889706 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888034 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-modprobe-d\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.889706 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888085 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qgtc\" (UniqueName: \"kubernetes.io/projected/5e4fedf3-163f-4a60-8177-eb75bd1950c3-kube-api-access-4qgtc\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.889706 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888094 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/70bb0a63-1beb-4f24-bcb6-e2a95ac1f429-iptables-alerter-script\") pod \"iptables-alerter-jbftz\" (UID: \"70bb0a63-1beb-4f24-bcb6-e2a95ac1f429\") " pod="openshift-network-operator/iptables-alerter-jbftz" Jul 09 01:59:40.889706 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888134 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec9d7c46-2a09-420b-90ae-5aad678928b0-ovnkube-script-lib\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.889706 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888143 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-multus-cni-dir\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.889706 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888173 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-host-run-multus-certs\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.889706 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888199 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-sys\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.889706 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888219 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-multus-cni-dir\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.889706 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888225 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-lib-modules\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.889706 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888253 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret\") pod \"global-pull-secret-syncer-lg95f\" (UID: \"7d15267e-8d0e-49e2-9a69-9c85f84b645f\") " pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:40.889706 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888286 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/83da97a5-3688-4c0d-97b6-cc3cc3ed3d25-serviceca\") pod \"node-ca-qkgn2\" (UID: \"83da97a5-3688-4c0d-97b6-cc3cc3ed3d25\") " pod="openshift-image-registry/node-ca-qkgn2" Jul 09 01:59:40.889706 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888303 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-sys\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.889706 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888312 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db97ff5f-15c4-4cef-b16c-fa03e39df587-system-cni-dir\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.889706 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888342 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-multus-socket-dir-parent\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.889706 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888348 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-host-run-multus-certs\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.890618 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888354 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db97ff5f-15c4-4cef-b16c-fa03e39df587-cni-binary-copy\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.890618 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888373 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-slash\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.890618 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888396 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db97ff5f-15c4-4cef-b16c-fa03e39df587-system-cni-dir\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.890618 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888399 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-etc-openvswitch\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.890618 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888416 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-lib-modules\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.890618 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:40.888409 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 01:59:40.890618 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888444 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-multus-socket-dir-parent\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.890618 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888456 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-etc-openvswitch\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.890618 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888459 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-slash\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.890618 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:40.888485 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret podName:7d15267e-8d0e-49e2-9a69-9c85f84b645f nodeName:}" failed. No retries permitted until 2026-07-09 01:59:41.388468005 +0000 UTC m=+2.086521804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret") pod "global-pull-secret-syncer-lg95f" (UID: "7d15267e-8d0e-49e2-9a69-9c85f84b645f") : object "kube-system"/"original-pull-secret" not registered Jul 09 01:59:40.890618 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888520 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1d33222e-bef7-4933-bfed-1a0ed60d9798-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.890618 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888550 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzggw\" (UniqueName: \"kubernetes.io/projected/db97ff5f-15c4-4cef-b16c-fa03e39df587-kube-api-access-tzggw\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.890618 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888612 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e4fedf3-163f-4a60-8177-eb75bd1950c3-tmp\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.890618 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888660 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-host-var-lib-cni-multus\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.890618 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888670 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1d33222e-bef7-4933-bfed-1a0ed60d9798-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.890618 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888689 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-run-netns\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.890618 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888715 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-node-log\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.891258 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888749 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec9d7c46-2a09-420b-90ae-5aad678928b0-env-overrides\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.891258 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888769 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/83da97a5-3688-4c0d-97b6-cc3cc3ed3d25-host\") pod \"node-ca-qkgn2\" (UID: \"83da97a5-3688-4c0d-97b6-cc3cc3ed3d25\") " pod="openshift-image-registry/node-ca-qkgn2" Jul 09 01:59:40.891258 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888789 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a403528-9b59-4685-974e-69e52cac4ec4-hosts-file\") pod \"node-resolver-bb6hq\" (UID: \"2a403528-9b59-4685-974e-69e52cac4ec4\") " pod="openshift-dns/node-resolver-bb6hq" Jul 09 01:59:40.891258 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888795 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-run-netns\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.891258 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888813 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-multus-daemon-config\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.891258 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888882 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-host-var-lib-cni-multus\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.891258 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888888 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xrqm\" (UniqueName: \"kubernetes.io/projected/70bb0a63-1beb-4f24-bcb6-e2a95ac1f429-kube-api-access-2xrqm\") pod \"iptables-alerter-jbftz\" (UID: \"70bb0a63-1beb-4f24-bcb6-e2a95ac1f429\") " pod="openshift-network-operator/iptables-alerter-jbftz" Jul 09 01:59:40.891258 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888944 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-kubelet\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.891258 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.888994 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-var-lib-openvswitch\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.891258 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889022 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-run-openvswitch\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.891258 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889048 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-sysconfig\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.891258 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889074 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/70bb0a63-1beb-4f24-bcb6-e2a95ac1f429-host-slash\") pod \"iptables-alerter-jbftz\" (UID: \"70bb0a63-1beb-4f24-bcb6-e2a95ac1f429\") " pod="openshift-network-operator/iptables-alerter-jbftz" Jul 09 01:59:40.891258 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889098 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ae3b2d7f-4db7-486c-b226-b37e4cfab1f8-konnectivity-ca\") pod \"konnectivity-agent-k9d84\" (UID: \"ae3b2d7f-4db7-486c-b226-b37e4cfab1f8\") " pod="kube-system/konnectivity-agent-k9d84" Jul 09 01:59:40.891258 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889122 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-multus-conf-dir\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.891258 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889146 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-log-socket\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.891258 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889166 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-node-log\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.891258 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889171 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec9d7c46-2a09-420b-90ae-5aad678928b0-ovnkube-config\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.891258 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889214 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1d33222e-bef7-4933-bfed-1a0ed60d9798-socket-dir\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.891809 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889231 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/83da97a5-3688-4c0d-97b6-cc3cc3ed3d25-host\") pod \"node-ca-qkgn2\" (UID: \"83da97a5-3688-4c0d-97b6-cc3cc3ed3d25\") " pod="openshift-image-registry/node-ca-qkgn2" Jul 09 01:59:40.891809 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889241 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-kubernetes\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.891809 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889330 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-kubernetes\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.891809 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889347 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 01:59:40.891809 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889373 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-host-kubelet\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.891809 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889411 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-var-lib-openvswitch\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.891809 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889490 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-run-openvswitch\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.891809 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889543 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-sysconfig\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.891809 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889593 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/70bb0a63-1beb-4f24-bcb6-e2a95ac1f429-host-slash\") pod \"iptables-alerter-jbftz\" (UID: \"70bb0a63-1beb-4f24-bcb6-e2a95ac1f429\") " pod="openshift-network-operator/iptables-alerter-jbftz" Jul 09 01:59:40.891809 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889607 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec9d7c46-2a09-420b-90ae-5aad678928b0-env-overrides\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.891809 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889741 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec9d7c46-2a09-420b-90ae-5aad678928b0-log-socket\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.891809 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.889792 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-multus-conf-dir\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.891809 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.890048 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5e4fedf3-163f-4a60-8177-eb75bd1950c3-etc-tuned\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.891809 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.890172 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec9d7c46-2a09-420b-90ae-5aad678928b0-ovn-node-metrics-cert\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.891809 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.890180 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-multus-daemon-config\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.891809 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.890296 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1d33222e-bef7-4933-bfed-1a0ed60d9798-socket-dir\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.891809 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.890311 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ae3b2d7f-4db7-486c-b226-b37e4cfab1f8-konnectivity-ca\") pod \"konnectivity-agent-k9d84\" (UID: \"ae3b2d7f-4db7-486c-b226-b37e4cfab1f8\") " pod="kube-system/konnectivity-agent-k9d84" Jul 09 01:59:40.891809 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.890389 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a403528-9b59-4685-974e-69e52cac4ec4-hosts-file\") pod \"node-resolver-bb6hq\" (UID: \"2a403528-9b59-4685-974e-69e52cac4ec4\") " pod="openshift-dns/node-resolver-bb6hq" Jul 09 01:59:40.892369 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.890522 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec9d7c46-2a09-420b-90ae-5aad678928b0-ovnkube-config\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.892369 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.890796 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ae3b2d7f-4db7-486c-b226-b37e4cfab1f8-agent-certs\") pod \"konnectivity-agent-k9d84\" (UID: \"ae3b2d7f-4db7-486c-b226-b37e4cfab1f8\") " pod="kube-system/konnectivity-agent-k9d84" Jul 09 01:59:40.892369 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.891378 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e4fedf3-163f-4a60-8177-eb75bd1950c3-tmp\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.901408 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.901362 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtdsr\" (UniqueName: \"kubernetes.io/projected/83da97a5-3688-4c0d-97b6-cc3cc3ed3d25-kube-api-access-qtdsr\") pod \"node-ca-qkgn2\" (UID: \"83da97a5-3688-4c0d-97b6-cc3cc3ed3d25\") " pod="openshift-image-registry/node-ca-qkgn2" Jul 09 01:59:40.908486 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.908461 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t64bp\" (UniqueName: \"kubernetes.io/projected/ec9d7c46-2a09-420b-90ae-5aad678928b0-kube-api-access-t64bp\") pod \"ovnkube-node-lj28w\" (UID: \"ec9d7c46-2a09-420b-90ae-5aad678928b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:40.908818 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.908801 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnmsq\" (UniqueName: \"kubernetes.io/projected/1d33222e-bef7-4933-bfed-1a0ed60d9798-kube-api-access-pnmsq\") pod \"aws-ebs-csi-driver-node-6pxcm\" (UID: \"1d33222e-bef7-4933-bfed-1a0ed60d9798\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:40.910973 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.910746 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2qkc\" (UniqueName: \"kubernetes.io/projected/2a403528-9b59-4685-974e-69e52cac4ec4-kube-api-access-s2qkc\") pod \"node-resolver-bb6hq\" (UID: \"2a403528-9b59-4685-974e-69e52cac4ec4\") " pod="openshift-dns/node-resolver-bb6hq" Jul 09 01:59:40.913865 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.913843 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9x76\" (UniqueName: \"kubernetes.io/projected/c5e07105-f26f-4ea9-8cde-9db06c456edc-kube-api-access-k9x76\") pod \"network-metrics-daemon-trtrf\" (UID: \"c5e07105-f26f-4ea9-8cde-9db06c456edc\") " pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:40.914005 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:40.913909 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 01:59:40.914005 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:40.913928 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 01:59:40.914005 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:40.913942 2582 projected.go:196] Error preparing data for projected volume kube-api-access-wmh2k for pod openshift-network-diagnostics/network-check-target-vgz69: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 01:59:40.914173 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:40.914121 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k podName:1941492f-5a14-4095-8743-70cb9e39ebe1 nodeName:}" failed. No retries permitted until 2026-07-09 01:59:41.414103801 +0000 UTC m=+2.112157594 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wmh2k" (UniqueName: "kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k") pod "network-check-target-vgz69" (UID: "1941492f-5a14-4095-8743-70cb9e39ebe1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 01:59:40.915895 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.915872 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzggw\" (UniqueName: \"kubernetes.io/projected/db97ff5f-15c4-4cef-b16c-fa03e39df587-kube-api-access-tzggw\") pod \"multus-additional-cni-plugins-6hktj\" (UID: \"db97ff5f-15c4-4cef-b16c-fa03e39df587\") " pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:40.916254 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.916235 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9h9h\" (UniqueName: \"kubernetes.io/projected/b3da9cb9-8c08-45b2-ad3e-5c2936c513a7-kube-api-access-k9h9h\") pod \"multus-mmnsx\" (UID: \"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7\") " pod="openshift-multus/multus-mmnsx" Jul 09 01:59:40.916325 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.916265 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xrqm\" (UniqueName: \"kubernetes.io/projected/70bb0a63-1beb-4f24-bcb6-e2a95ac1f429-kube-api-access-2xrqm\") pod \"iptables-alerter-jbftz\" (UID: \"70bb0a63-1beb-4f24-bcb6-e2a95ac1f429\") " pod="openshift-network-operator/iptables-alerter-jbftz" Jul 09 01:59:40.916921 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.916901 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qgtc\" (UniqueName: \"kubernetes.io/projected/5e4fedf3-163f-4a60-8177-eb75bd1950c3-kube-api-access-4qgtc\") pod \"tuned-w24jg\" (UID: \"5e4fedf3-163f-4a60-8177-eb75bd1950c3\") " pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:40.920331 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:40.920316 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 01:59:41.009340 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.009283 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 01:59:41.018157 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:41.018138 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec9d7c46_2a09_420b_90ae_5aad678928b0.slice/crio-a06754f314cfa09bad489f505bf282f6651e7cbdb6a00c51ea789fb960b504be WatchSource:0}: Error finding container a06754f314cfa09bad489f505bf282f6651e7cbdb6a00c51ea789fb960b504be: Status 404 returned error can't find the container with id a06754f314cfa09bad489f505bf282f6651e7cbdb6a00c51ea789fb960b504be Jul 09 01:59:41.040672 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.040648 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w24jg" Jul 09 01:59:41.047663 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:41.047643 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e4fedf3_163f_4a60_8177_eb75bd1950c3.slice/crio-7f4872a3b8031755864559f2cf17485bfcb6b9baede0d5a3902576cddad542b9 WatchSource:0}: Error finding container 7f4872a3b8031755864559f2cf17485bfcb6b9baede0d5a3902576cddad542b9: Status 404 returned error can't find the container with id 7f4872a3b8031755864559f2cf17485bfcb6b9baede0d5a3902576cddad542b9 Jul 09 01:59:41.055134 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.055118 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" Jul 09 01:59:41.062607 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:41.062587 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d33222e_bef7_4933_bfed_1a0ed60d9798.slice/crio-8ef421f90e3057f88a35164879518d0074bced97d498b8851f03a671e99f8ea8 WatchSource:0}: Error finding container 8ef421f90e3057f88a35164879518d0074bced97d498b8851f03a671e99f8ea8: Status 404 returned error can't find the container with id 8ef421f90e3057f88a35164879518d0074bced97d498b8851f03a671e99f8ea8 Jul 09 01:59:41.073204 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.073181 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qkgn2" Jul 09 01:59:41.079913 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:41.079865 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83da97a5_3688_4c0d_97b6_cc3cc3ed3d25.slice/crio-d40395aaca94b4ce380fd40600d5cb0cf54b379bee391e56bd107738b2f8830d WatchSource:0}: Error finding container d40395aaca94b4ce380fd40600d5cb0cf54b379bee391e56bd107738b2f8830d: Status 404 returned error can't find the container with id d40395aaca94b4ce380fd40600d5cb0cf54b379bee391e56bd107738b2f8830d Jul 09 01:59:41.089756 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.089739 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6hktj" Jul 09 01:59:41.095434 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.095420 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k9d84" Jul 09 01:59:41.096260 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:41.096243 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb97ff5f_15c4_4cef_b16c_fa03e39df587.slice/crio-7fe213c71b6c4e5513f69cec18da55e4da3f467f67b4c6c39634780c9b193ae4 WatchSource:0}: Error finding container 7fe213c71b6c4e5513f69cec18da55e4da3f467f67b4c6c39634780c9b193ae4: Status 404 returned error can't find the container with id 7fe213c71b6c4e5513f69cec18da55e4da3f467f67b4c6c39634780c9b193ae4 Jul 09 01:59:41.102501 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:41.102483 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae3b2d7f_4db7_486c_b226_b37e4cfab1f8.slice/crio-a4b58404cc204b493b349b6155b3f2653ad40cef462f4c3231c43ded20464cba WatchSource:0}: Error finding container a4b58404cc204b493b349b6155b3f2653ad40cef462f4c3231c43ded20464cba: Status 404 returned error can't find the container with id a4b58404cc204b493b349b6155b3f2653ad40cef462f4c3231c43ded20464cba Jul 09 01:59:41.125263 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.125240 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bb6hq" Jul 09 01:59:41.130482 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.130466 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mmnsx" Jul 09 01:59:41.132308 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.132284 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jbftz" Jul 09 01:59:41.132400 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:41.132336 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a403528_9b59_4685_974e_69e52cac4ec4.slice/crio-bcf769d3bb0e4629714560bd566c355ebcd2923085dba764fa64ac0374557737 WatchSource:0}: Error finding container bcf769d3bb0e4629714560bd566c355ebcd2923085dba764fa64ac0374557737: Status 404 returned error can't find the container with id bcf769d3bb0e4629714560bd566c355ebcd2923085dba764fa64ac0374557737 Jul 09 01:59:41.138906 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:41.138889 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3da9cb9_8c08_45b2_ad3e_5c2936c513a7.slice/crio-f2789d5ef428645247f45f360fa046709c0da832da9c296e7b6653cd91395de5 WatchSource:0}: Error finding container f2789d5ef428645247f45f360fa046709c0da832da9c296e7b6653cd91395de5: Status 404 returned error can't find the container with id f2789d5ef428645247f45f360fa046709c0da832da9c296e7b6653cd91395de5 Jul 09 01:59:41.140863 ip-10-0-130-184 kubenswrapper[2582]: W0709 01:59:41.140839 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70bb0a63_1beb_4f24_bcb6_e2a95ac1f429.slice/crio-78a21ccecf6473dd00d66e033d1155ec709930c6df0fa0693e7c1959f205bcb6 WatchSource:0}: Error finding container 78a21ccecf6473dd00d66e033d1155ec709930c6df0fa0693e7c1959f205bcb6: Status 404 returned error can't find the container with id 78a21ccecf6473dd00d66e033d1155ec709930c6df0fa0693e7c1959f205bcb6 Jul 09 01:59:41.391190 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.391088 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs\") pod \"network-metrics-daemon-trtrf\" (UID: \"c5e07105-f26f-4ea9-8cde-9db06c456edc\") " pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:41.391190 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.391160 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret\") pod \"global-pull-secret-syncer-lg95f\" (UID: \"7d15267e-8d0e-49e2-9a69-9c85f84b645f\") " pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:41.391413 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:41.391231 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 01:59:41.391413 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:41.391281 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 01:59:41.391413 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:41.391295 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs podName:c5e07105-f26f-4ea9-8cde-9db06c456edc nodeName:}" failed. No retries permitted until 2026-07-09 01:59:42.391275629 +0000 UTC m=+3.089329447 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs") pod "network-metrics-daemon-trtrf" (UID: "c5e07105-f26f-4ea9-8cde-9db06c456edc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 01:59:41.391413 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:41.391318 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret podName:7d15267e-8d0e-49e2-9a69-9c85f84b645f nodeName:}" failed. No retries permitted until 2026-07-09 01:59:42.391306996 +0000 UTC m=+3.089360791 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret") pod "global-pull-secret-syncer-lg95f" (UID: "7d15267e-8d0e-49e2-9a69-9c85f84b645f") : object "kube-system"/"original-pull-secret" not registered Jul 09 01:59:41.492042 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.491980 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmh2k\" (UniqueName: \"kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k\") pod \"network-check-target-vgz69\" (UID: \"1941492f-5a14-4095-8743-70cb9e39ebe1\") " pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 01:59:41.492216 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:41.492175 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 01:59:41.492216 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:41.492202 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 01:59:41.492216 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:41.492216 2582 projected.go:196] Error preparing data for projected volume kube-api-access-wmh2k for pod openshift-network-diagnostics/network-check-target-vgz69: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 01:59:41.492403 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:41.492277 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k podName:1941492f-5a14-4095-8743-70cb9e39ebe1 nodeName:}" failed. No retries permitted until 2026-07-09 01:59:42.492256101 +0000 UTC m=+3.190309895 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wmh2k" (UniqueName: "kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k") pod "network-check-target-vgz69" (UID: "1941492f-5a14-4095-8743-70cb9e39ebe1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 01:59:41.770421 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.770261 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 01:54:41 +0000 UTC" deadline="2028-01-07 02:21:56.834096341 +0000 UTC" Jul 09 01:59:41.770421 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.770303 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13128h22m15.063798887s" Jul 09 01:59:41.862923 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.862386 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:41.862923 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:41.862525 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 01:59:41.882011 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.881919 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jbftz" event={"ID":"70bb0a63-1beb-4f24-bcb6-e2a95ac1f429","Type":"ContainerStarted","Data":"78a21ccecf6473dd00d66e033d1155ec709930c6df0fa0693e7c1959f205bcb6"} Jul 09 01:59:41.883837 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.883775 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bb6hq" event={"ID":"2a403528-9b59-4685-974e-69e52cac4ec4","Type":"ContainerStarted","Data":"bcf769d3bb0e4629714560bd566c355ebcd2923085dba764fa64ac0374557737"} Jul 09 01:59:41.891455 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.891383 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hktj" event={"ID":"db97ff5f-15c4-4cef-b16c-fa03e39df587","Type":"ContainerStarted","Data":"7fe213c71b6c4e5513f69cec18da55e4da3f467f67b4c6c39634780c9b193ae4"} Jul 09 01:59:41.900013 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.899802 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 01:59:41.901182 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.901109 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qkgn2" event={"ID":"83da97a5-3688-4c0d-97b6-cc3cc3ed3d25","Type":"ContainerStarted","Data":"d40395aaca94b4ce380fd40600d5cb0cf54b379bee391e56bd107738b2f8830d"} Jul 09 01:59:41.932902 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.932845 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" event={"ID":"1d33222e-bef7-4933-bfed-1a0ed60d9798","Type":"ContainerStarted","Data":"8ef421f90e3057f88a35164879518d0074bced97d498b8851f03a671e99f8ea8"} Jul 09 01:59:41.946237 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.946115 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" event={"ID":"ec9d7c46-2a09-420b-90ae-5aad678928b0","Type":"ContainerStarted","Data":"a06754f314cfa09bad489f505bf282f6651e7cbdb6a00c51ea789fb960b504be"} Jul 09 01:59:41.963332 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.962910 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mmnsx" event={"ID":"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7","Type":"ContainerStarted","Data":"f2789d5ef428645247f45f360fa046709c0da832da9c296e7b6653cd91395de5"} Jul 09 01:59:41.986211 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:41.986181 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k9d84" event={"ID":"ae3b2d7f-4db7-486c-b226-b37e4cfab1f8","Type":"ContainerStarted","Data":"a4b58404cc204b493b349b6155b3f2653ad40cef462f4c3231c43ded20464cba"} Jul 09 01:59:42.000616 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:42.000543 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w24jg" event={"ID":"5e4fedf3-163f-4a60-8177-eb75bd1950c3","Type":"ContainerStarted","Data":"7f4872a3b8031755864559f2cf17485bfcb6b9baede0d5a3902576cddad542b9"} Jul 09 01:59:42.402106 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:42.401267 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret\") pod \"global-pull-secret-syncer-lg95f\" (UID: \"7d15267e-8d0e-49e2-9a69-9c85f84b645f\") " pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:42.402106 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:42.401359 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs\") pod \"network-metrics-daemon-trtrf\" (UID: \"c5e07105-f26f-4ea9-8cde-9db06c456edc\") " pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:42.402106 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:42.401502 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 01:59:42.402106 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:42.401568 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs podName:c5e07105-f26f-4ea9-8cde-9db06c456edc nodeName:}" failed. No retries permitted until 2026-07-09 01:59:44.401547383 +0000 UTC m=+5.099601182 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs") pod "network-metrics-daemon-trtrf" (UID: "c5e07105-f26f-4ea9-8cde-9db06c456edc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 01:59:42.402106 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:42.402011 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 01:59:42.402106 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:42.402068 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret podName:7d15267e-8d0e-49e2-9a69-9c85f84b645f nodeName:}" failed. No retries permitted until 2026-07-09 01:59:44.402051357 +0000 UTC m=+5.100105152 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret") pod "global-pull-secret-syncer-lg95f" (UID: "7d15267e-8d0e-49e2-9a69-9c85f84b645f") : object "kube-system"/"original-pull-secret" not registered Jul 09 01:59:42.503718 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:42.502347 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmh2k\" (UniqueName: \"kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k\") pod \"network-check-target-vgz69\" (UID: \"1941492f-5a14-4095-8743-70cb9e39ebe1\") " pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 01:59:42.503718 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:42.502513 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 01:59:42.503718 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:42.502530 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 01:59:42.503718 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:42.502544 2582 projected.go:196] Error preparing data for projected volume kube-api-access-wmh2k for pod openshift-network-diagnostics/network-check-target-vgz69: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 01:59:42.503718 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:42.502598 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k podName:1941492f-5a14-4095-8743-70cb9e39ebe1 nodeName:}" failed. No retries permitted until 2026-07-09 01:59:44.502580362 +0000 UTC m=+5.200634159 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wmh2k" (UniqueName: "kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k") pod "network-check-target-vgz69" (UID: "1941492f-5a14-4095-8743-70cb9e39ebe1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 01:59:42.771236 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:42.771126 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 01:54:41 +0000 UTC" deadline="2028-04-01 22:52:11.606928314 +0000 UTC" Jul 09 01:59:42.771236 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:42.771186 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15188h52m28.835746361s" Jul 09 01:59:42.860256 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:42.860224 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:42.860418 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:42.860383 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trtrf" podUID="c5e07105-f26f-4ea9-8cde-9db06c456edc" Jul 09 01:59:42.860820 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:42.860801 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 01:59:42.860926 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:42.860905 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vgz69" podUID="1941492f-5a14-4095-8743-70cb9e39ebe1" Jul 09 01:59:43.313419 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:43.313350 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 01:59:43.860737 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:43.860246 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:43.860737 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:43.860383 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 01:59:44.420436 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:44.420394 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs\") pod \"network-metrics-daemon-trtrf\" (UID: \"c5e07105-f26f-4ea9-8cde-9db06c456edc\") " pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:44.420654 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:44.420483 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret\") pod \"global-pull-secret-syncer-lg95f\" (UID: \"7d15267e-8d0e-49e2-9a69-9c85f84b645f\") " pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:44.420654 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:44.420625 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 01:59:44.420771 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:44.420689 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret podName:7d15267e-8d0e-49e2-9a69-9c85f84b645f nodeName:}" failed. No retries permitted until 2026-07-09 01:59:48.420670369 +0000 UTC m=+9.118724165 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret") pod "global-pull-secret-syncer-lg95f" (UID: "7d15267e-8d0e-49e2-9a69-9c85f84b645f") : object "kube-system"/"original-pull-secret" not registered Jul 09 01:59:44.421237 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:44.421106 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 01:59:44.421237 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:44.421164 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs podName:c5e07105-f26f-4ea9-8cde-9db06c456edc nodeName:}" failed. No retries permitted until 2026-07-09 01:59:48.421146409 +0000 UTC m=+9.119200206 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs") pod "network-metrics-daemon-trtrf" (UID: "c5e07105-f26f-4ea9-8cde-9db06c456edc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 01:59:44.520903 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:44.520856 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmh2k\" (UniqueName: \"kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k\") pod \"network-check-target-vgz69\" (UID: \"1941492f-5a14-4095-8743-70cb9e39ebe1\") " pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 01:59:44.521131 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:44.521055 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 01:59:44.521131 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:44.521080 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 01:59:44.521131 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:44.521094 2582 projected.go:196] Error preparing data for projected volume kube-api-access-wmh2k for pod openshift-network-diagnostics/network-check-target-vgz69: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 01:59:44.521335 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:44.521166 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k podName:1941492f-5a14-4095-8743-70cb9e39ebe1 nodeName:}" failed. No retries permitted until 2026-07-09 01:59:48.521144257 +0000 UTC m=+9.219198054 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wmh2k" (UniqueName: "kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k") pod "network-check-target-vgz69" (UID: "1941492f-5a14-4095-8743-70cb9e39ebe1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 01:59:44.860305 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:44.860224 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:44.860454 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:44.860334 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trtrf" podUID="c5e07105-f26f-4ea9-8cde-9db06c456edc" Jul 09 01:59:44.860565 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:44.860542 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 01:59:44.860702 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:44.860676 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vgz69" podUID="1941492f-5a14-4095-8743-70cb9e39ebe1" Jul 09 01:59:45.859370 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:45.859331 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:45.859859 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:45.859490 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 01:59:46.860444 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:46.860391 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 01:59:46.860979 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:46.860546 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vgz69" podUID="1941492f-5a14-4095-8743-70cb9e39ebe1" Jul 09 01:59:46.860979 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:46.860624 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:46.860979 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:46.860760 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trtrf" podUID="c5e07105-f26f-4ea9-8cde-9db06c456edc" Jul 09 01:59:47.859770 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:47.859674 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:47.859954 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:47.859811 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 01:59:48.458172 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:48.458135 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs\") pod \"network-metrics-daemon-trtrf\" (UID: \"c5e07105-f26f-4ea9-8cde-9db06c456edc\") " pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:48.458172 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:48.458206 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret\") pod \"global-pull-secret-syncer-lg95f\" (UID: \"7d15267e-8d0e-49e2-9a69-9c85f84b645f\") " pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:48.458742 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:48.458358 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 01:59:48.458742 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:48.458420 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret podName:7d15267e-8d0e-49e2-9a69-9c85f84b645f nodeName:}" failed. No retries permitted until 2026-07-09 01:59:56.458400745 +0000 UTC m=+17.156454541 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret") pod "global-pull-secret-syncer-lg95f" (UID: "7d15267e-8d0e-49e2-9a69-9c85f84b645f") : object "kube-system"/"original-pull-secret" not registered Jul 09 01:59:48.458742 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:48.458582 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 01:59:48.458742 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:48.458652 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs podName:c5e07105-f26f-4ea9-8cde-9db06c456edc nodeName:}" failed. No retries permitted until 2026-07-09 01:59:56.458632848 +0000 UTC m=+17.156686655 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs") pod "network-metrics-daemon-trtrf" (UID: "c5e07105-f26f-4ea9-8cde-9db06c456edc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 01:59:48.559026 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:48.558870 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmh2k\" (UniqueName: \"kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k\") pod \"network-check-target-vgz69\" (UID: \"1941492f-5a14-4095-8743-70cb9e39ebe1\") " pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 01:59:48.559200 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:48.559057 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 01:59:48.559200 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:48.559079 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 01:59:48.559200 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:48.559089 2582 projected.go:196] Error preparing data for projected volume kube-api-access-wmh2k for pod openshift-network-diagnostics/network-check-target-vgz69: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 01:59:48.559200 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:48.559150 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k podName:1941492f-5a14-4095-8743-70cb9e39ebe1 nodeName:}" failed. No retries permitted until 2026-07-09 01:59:56.559133997 +0000 UTC m=+17.257187791 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wmh2k" (UniqueName: "kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k") pod "network-check-target-vgz69" (UID: "1941492f-5a14-4095-8743-70cb9e39ebe1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 01:59:48.860131 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:48.860044 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:48.860294 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:48.860205 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trtrf" podUID="c5e07105-f26f-4ea9-8cde-9db06c456edc" Jul 09 01:59:48.860704 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:48.860549 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 01:59:48.860704 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:48.860649 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vgz69" podUID="1941492f-5a14-4095-8743-70cb9e39ebe1" Jul 09 01:59:49.861043 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:49.860729 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:49.861043 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:49.860844 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 01:59:50.859469 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:50.859436 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:50.859739 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:50.859436 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 01:59:50.859739 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:50.859585 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trtrf" podUID="c5e07105-f26f-4ea9-8cde-9db06c456edc" Jul 09 01:59:50.859739 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:50.859635 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vgz69" podUID="1941492f-5a14-4095-8743-70cb9e39ebe1" Jul 09 01:59:51.859998 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:51.859966 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:51.860444 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:51.860094 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 01:59:52.859626 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:52.859599 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 01:59:52.859764 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:52.859600 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:52.859764 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:52.859688 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vgz69" podUID="1941492f-5a14-4095-8743-70cb9e39ebe1" Jul 09 01:59:52.859888 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:52.859771 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trtrf" podUID="c5e07105-f26f-4ea9-8cde-9db06c456edc" Jul 09 01:59:53.859926 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:53.859900 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:53.860366 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:53.860014 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 01:59:54.859494 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:54.859450 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 01:59:54.859494 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:54.859503 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:54.859770 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:54.859606 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vgz69" podUID="1941492f-5a14-4095-8743-70cb9e39ebe1" Jul 09 01:59:54.859770 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:54.859688 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trtrf" podUID="c5e07105-f26f-4ea9-8cde-9db06c456edc" Jul 09 01:59:55.860371 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:55.860171 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:55.860775 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:55.860466 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 01:59:56.519680 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:56.519649 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs\") pod \"network-metrics-daemon-trtrf\" (UID: \"c5e07105-f26f-4ea9-8cde-9db06c456edc\") " pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:56.519924 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:56.519713 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret\") pod \"global-pull-secret-syncer-lg95f\" (UID: \"7d15267e-8d0e-49e2-9a69-9c85f84b645f\") " pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:56.519924 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:56.519809 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 01:59:56.519924 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:56.519864 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs podName:c5e07105-f26f-4ea9-8cde-9db06c456edc nodeName:}" failed. No retries permitted until 2026-07-09 02:00:12.519849104 +0000 UTC m=+33.217902895 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs") pod "network-metrics-daemon-trtrf" (UID: "c5e07105-f26f-4ea9-8cde-9db06c456edc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 01:59:56.519924 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:56.519814 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 01:59:56.520115 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:56.519958 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret podName:7d15267e-8d0e-49e2-9a69-9c85f84b645f nodeName:}" failed. No retries permitted until 2026-07-09 02:00:12.519939871 +0000 UTC m=+33.217993677 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret") pod "global-pull-secret-syncer-lg95f" (UID: "7d15267e-8d0e-49e2-9a69-9c85f84b645f") : object "kube-system"/"original-pull-secret" not registered Jul 09 01:59:56.620486 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:56.620449 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmh2k\" (UniqueName: \"kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k\") pod \"network-check-target-vgz69\" (UID: \"1941492f-5a14-4095-8743-70cb9e39ebe1\") " pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 01:59:56.620679 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:56.620637 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 01:59:56.620679 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:56.620667 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 01:59:56.620679 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:56.620682 2582 projected.go:196] Error preparing data for projected volume kube-api-access-wmh2k for pod openshift-network-diagnostics/network-check-target-vgz69: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 01:59:56.620841 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:56.620743 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k podName:1941492f-5a14-4095-8743-70cb9e39ebe1 nodeName:}" failed. No retries permitted until 2026-07-09 02:00:12.620724705 +0000 UTC m=+33.318778512 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wmh2k" (UniqueName: "kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k") pod "network-check-target-vgz69" (UID: "1941492f-5a14-4095-8743-70cb9e39ebe1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 01:59:56.859624 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:56.859554 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:56.859782 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:56.859668 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trtrf" podUID="c5e07105-f26f-4ea9-8cde-9db06c456edc" Jul 09 01:59:56.859782 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:56.859719 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 01:59:56.859851 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:56.859799 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vgz69" podUID="1941492f-5a14-4095-8743-70cb9e39ebe1" Jul 09 01:59:57.859989 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:57.859951 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:57.860489 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:57.860091 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 01:59:58.860207 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:58.860182 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 01:59:58.860583 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:58.860298 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vgz69" podUID="1941492f-5a14-4095-8743-70cb9e39ebe1" Jul 09 01:59:58.860583 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:58.860348 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 01:59:58.860583 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:58.860456 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trtrf" podUID="c5e07105-f26f-4ea9-8cde-9db06c456edc" Jul 09 01:59:59.860082 ip-10-0-130-184 kubenswrapper[2582]: I0709 01:59:59.860049 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 01:59:59.860261 ip-10-0-130-184 kubenswrapper[2582]: E0709 01:59:59.860148 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 02:00:00.860259 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:00.860226 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 02:00:00.860688 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:00.860225 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 02:00:00.860688 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:00.860360 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vgz69" podUID="1941492f-5a14-4095-8743-70cb9e39ebe1" Jul 09 02:00:00.860688 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:00.860461 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trtrf" podUID="c5e07105-f26f-4ea9-8cde-9db06c456edc" Jul 09 02:00:01.860186 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:01.860117 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 02:00:01.860317 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:01.860232 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 02:00:02.052367 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:02.052335 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-184.ec2.internal" event={"ID":"81d8756a43625938ca8b36e75b20e52a","Type":"ContainerStarted","Data":"fcafdd178d7313e22f3e8e996745d41ac4a14dbcfde3dbf934ebfb146e39a99d"} Jul 09 02:00:02.053789 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:02.053763 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mmnsx" event={"ID":"b3da9cb9-8c08-45b2-ad3e-5c2936c513a7","Type":"ContainerStarted","Data":"0e61faeb747e79504d8bcf85e716bbc7918a1bb7b4b82b80962cb77e66f7ba7b"} Jul 09 02:00:02.055212 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:02.055185 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w24jg" event={"ID":"5e4fedf3-163f-4a60-8177-eb75bd1950c3","Type":"ContainerStarted","Data":"ee6a34e84acbc500a6eabb35a73ac044333fb75f661a0eadc36763d3b2261fa9"} Jul 09 02:00:02.057231 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:02.057209 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" event={"ID":"ec9d7c46-2a09-420b-90ae-5aad678928b0","Type":"ContainerStarted","Data":"bd2be3d1f9d0d76cc3e47e343448986d7d6b00e690c0ad2806ea797582f6f7da"} Jul 09 02:00:02.066070 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:02.066050 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-184.ec2.internal" Jul 09 02:00:02.081716 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:02.081693 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 02:00:02.085060 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:02.083121 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-184.ec2.internal"] Jul 09 02:00:02.091116 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:02.091056 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mmnsx" podStartSLOduration=1.839966434 podStartE2EDuration="22.091039955s" totalImagesPullingTime="20.251073521s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 01:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:02.09072893 +0000 UTC m=+22.788782743" watchObservedRunningTime="2026-07-09 02:00:02.091039955 +0000 UTC m=+22.789093765" Jul 09 02:00:02.107046 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:02.107005 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-184.ec2.internal" podStartSLOduration=0.10698821 podStartE2EDuration="106.98821ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 02:00:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:02.10627325 +0000 UTC m=+22.804327067" watchObservedRunningTime="2026-07-09 02:00:02.10698821 +0000 UTC m=+22.805042028" Jul 09 02:00:02.138608 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:02.138568 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-w24jg" podStartSLOduration=1.804483277 podStartE2EDuration="22.13855401s" totalImagesPullingTime="20.334070733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 01:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:02.138432448 +0000 UTC m=+22.836486260" watchObservedRunningTime="2026-07-09 02:00:02.13855401 +0000 UTC m=+22.836607813" Jul 09 02:00:02.860139 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:02.859846 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 02:00:02.860227 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:02.859918 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 02:00:02.860227 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:02.860218 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trtrf" podUID="c5e07105-f26f-4ea9-8cde-9db06c456edc" Jul 09 02:00:02.860308 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:02.860286 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vgz69" podUID="1941492f-5a14-4095-8743-70cb9e39ebe1" Jul 09 02:00:03.060852 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.060733 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k9d84" event={"ID":"ae3b2d7f-4db7-486c-b226-b37e4cfab1f8","Type":"ContainerStarted","Data":"1a213603b375c1b703f2ce31cb734d19af355acc95930229f14b68b11edc850d"} Jul 09 02:00:03.062278 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.062257 2582 generic.go:356] "Generic (PLEG): container finished" podID="fa270ef1e2da30182fb1ce8e87a3f7c2" containerID="f5a24d789a9f2d1ef88d709f278737374166b7e71b36a8256100ccf4b6d46aa3" exitCode=0 Jul 09 02:00:03.062375 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.062334 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal" event={"ID":"fa270ef1e2da30182fb1ce8e87a3f7c2","Type":"ContainerDied","Data":"f5a24d789a9f2d1ef88d709f278737374166b7e71b36a8256100ccf4b6d46aa3"} Jul 09 02:00:03.063884 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.063843 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jbftz" event={"ID":"70bb0a63-1beb-4f24-bcb6-e2a95ac1f429","Type":"ContainerStarted","Data":"286de15c5aafff529893cac04742d45b726eb1ad7d91fc7082bee8566ebb7b4f"} Jul 09 02:00:03.065218 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.065200 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bb6hq" event={"ID":"2a403528-9b59-4685-974e-69e52cac4ec4","Type":"ContainerStarted","Data":"7a379829e613550293598355d98ea5542156ee9dd9391501e4c392988f186cf2"} Jul 09 02:00:03.066695 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.066673 2582 generic.go:356] "Generic (PLEG): container finished" podID="db97ff5f-15c4-4cef-b16c-fa03e39df587" containerID="e2fbda138ea7ec410c880b1d6a6eb5c512b5124621350580fa552d5eda8e78a4" exitCode=0 Jul 09 02:00:03.066800 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.066733 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hktj" event={"ID":"db97ff5f-15c4-4cef-b16c-fa03e39df587","Type":"ContainerDied","Data":"e2fbda138ea7ec410c880b1d6a6eb5c512b5124621350580fa552d5eda8e78a4"} Jul 09 02:00:03.068037 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.068021 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qkgn2" event={"ID":"83da97a5-3688-4c0d-97b6-cc3cc3ed3d25","Type":"ContainerStarted","Data":"40fb86c93e9b8797be9339c8f6043f433b3d9f8ba37f4303e4e20b83ce82829d"} Jul 09 02:00:03.069432 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.069413 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" event={"ID":"1d33222e-bef7-4933-bfed-1a0ed60d9798","Type":"ContainerStarted","Data":"e3f017d9491323fe4163f2229874eb052297dc51fcfc73c7d23b7b3ab542bfb3"} Jul 09 02:00:03.071729 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.071712 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:00:03.072033 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.072017 2582 generic.go:356] "Generic (PLEG): container finished" podID="ec9d7c46-2a09-420b-90ae-5aad678928b0" containerID="a6b6c328f8aed6829cb6f7037f8de43fbf31a1b3c654560cd9f02fbb994b03c9" exitCode=1 Jul 09 02:00:03.072122 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.072103 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" event={"ID":"ec9d7c46-2a09-420b-90ae-5aad678928b0","Type":"ContainerStarted","Data":"c583f655af818077c727be66198a0eab88996bf50052b6b45e541a33f84d0c82"} Jul 09 02:00:03.072168 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.072159 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" event={"ID":"ec9d7c46-2a09-420b-90ae-5aad678928b0","Type":"ContainerStarted","Data":"5d0a0875fc9db3131015213466ca47c33bd1fa1aed9771ca243eed4aff2f9f75"} Jul 09 02:00:03.072202 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.072190 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" event={"ID":"ec9d7c46-2a09-420b-90ae-5aad678928b0","Type":"ContainerStarted","Data":"9f7bcf830f995c123b634d8e51ee7e83629b4794697902639b683fa1d21af250"} Jul 09 02:00:03.072233 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.072222 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" event={"ID":"ec9d7c46-2a09-420b-90ae-5aad678928b0","Type":"ContainerStarted","Data":"3f87007c5deedca34efd920a0ed4bf2271cc9e0266594ba4498c9215fc7e539b"} Jul 09 02:00:03.072269 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.072253 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" event={"ID":"ec9d7c46-2a09-420b-90ae-5aad678928b0","Type":"ContainerDied","Data":"a6b6c328f8aed6829cb6f7037f8de43fbf31a1b3c654560cd9f02fbb994b03c9"} Jul 09 02:00:03.072391 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.072379 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-184.ec2.internal" Jul 09 02:00:03.082749 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.080945 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-k9d84" podStartSLOduration=2.827846177 podStartE2EDuration="23.080929613s" totalImagesPullingTime="20.253083436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 01:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:03.08067958 +0000 UTC m=+23.778733394" watchObservedRunningTime="2026-07-09 02:00:03.080929613 +0000 UTC m=+23.778983426" Jul 09 02:00:03.084000 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.083980 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 02:00:03.084083 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:03.084019 2582 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-130-184.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-184.ec2.internal" Jul 09 02:00:03.121908 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.121851 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jbftz" podStartSLOduration=2.879686491 podStartE2EDuration="23.121834208s" totalImagesPullingTime="20.242147717s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 01:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:03.121800629 +0000 UTC m=+23.819854443" watchObservedRunningTime="2026-07-09 02:00:03.121834208 +0000 UTC m=+23.819888024" Jul 09 02:00:03.157252 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.157206 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qkgn2" podStartSLOduration=2.855950887 podStartE2EDuration="23.157192444s" totalImagesPullingTime="20.301241557s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 01:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:03.156938783 +0000 UTC m=+23.854992595" watchObservedRunningTime="2026-07-09 02:00:03.157192444 +0000 UTC m=+23.855246257" Jul 09 02:00:03.177488 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.177451 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bb6hq" podStartSLOduration=11.248761958 podStartE2EDuration="23.177438761s" totalImagesPullingTime="11.928676803s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 01:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:03.17718909 +0000 UTC m=+23.875242905" watchObservedRunningTime="2026-07-09 02:00:03.177438761 +0000 UTC m=+23.875492573" Jul 09 02:00:03.859738 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:03.859706 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 02:00:03.860010 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:03.859816 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 02:00:04.077042 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:04.077007 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal" event={"ID":"fa270ef1e2da30182fb1ce8e87a3f7c2","Type":"ContainerStarted","Data":"8cf1cec1b8b4c52a5130683d8ffceaffb65b2e5854f3c79f0f1ac834a6ed01d1"} Jul 09 02:00:04.098481 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:04.098427 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-184.ec2.internal" podStartSLOduration=23.09840623 podStartE2EDuration="23.09840623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 01:59:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:04.098266867 +0000 UTC m=+24.796320682" watchObservedRunningTime="2026-07-09 02:00:04.09840623 +0000 UTC m=+24.796460045" Jul 09 02:00:04.689298 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:04.689269 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 02:00:04.753186 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:04.753089 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T02:00:04.689293447Z","UUID":"89afb533-6536-48db-b45d-f4c17619b4eb","Handler":null,"Name":"","Endpoint":""} Jul 09 02:00:04.755148 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:04.755124 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 02:00:04.755294 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:04.755170 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 02:00:04.860031 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:04.859957 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 02:00:04.860223 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:04.860080 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trtrf" podUID="c5e07105-f26f-4ea9-8cde-9db06c456edc" Jul 09 02:00:04.860223 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:04.860136 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 02:00:04.860341 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:04.860225 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vgz69" podUID="1941492f-5a14-4095-8743-70cb9e39ebe1" Jul 09 02:00:05.088250 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:05.088204 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" event={"ID":"1d33222e-bef7-4933-bfed-1a0ed60d9798","Type":"ContainerStarted","Data":"e5aeb8264c507c321b561dfdc6f6ee31d5af42f7ea1484edc7b2ea6d2abefdc9"} Jul 09 02:00:05.091775 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:05.091751 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:00:05.092239 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:05.092206 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" event={"ID":"ec9d7c46-2a09-420b-90ae-5aad678928b0","Type":"ContainerStarted","Data":"239c8fba98968235e17e268d2db66d5a51035e63ceecdfaa312de5eed3bdb7e3"} Jul 09 02:00:05.859603 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:05.859558 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 02:00:05.859950 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:05.859673 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 02:00:06.859655 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:06.859467 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 02:00:06.860121 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:06.859512 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 02:00:06.860121 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:06.859767 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trtrf" podUID="c5e07105-f26f-4ea9-8cde-9db06c456edc" Jul 09 02:00:06.860121 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:06.859804 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vgz69" podUID="1941492f-5a14-4095-8743-70cb9e39ebe1" Jul 09 02:00:07.099023 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:07.098934 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" event={"ID":"1d33222e-bef7-4933-bfed-1a0ed60d9798","Type":"ContainerStarted","Data":"b6f3879eb0acf7889273693425f77f7ab5ac5069a96088f00bb844f436452244"} Jul 09 02:00:07.122489 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:07.122437 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6pxcm" podStartSLOduration=3.264865269 podStartE2EDuration="27.122422865s" totalImagesPullingTime="23.857557596s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 01:59:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:07.12196993 +0000 UTC m=+27.820023733" watchObservedRunningTime="2026-07-09 02:00:07.122422865 +0000 UTC m=+27.820476677" Jul 09 02:00:07.372944 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:07.372857 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-k9d84" Jul 09 02:00:07.373543 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:07.373517 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-k9d84" Jul 09 02:00:07.860214 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:07.860143 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 02:00:07.860582 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:07.860241 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 02:00:08.101713 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:08.101622 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-k9d84" Jul 09 02:00:08.102078 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:08.102057 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-k9d84" Jul 09 02:00:08.223173 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:08.223035 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:08.380159 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:08.380101 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:08.860405 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:08.860167 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 02:00:08.860405 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:08.860167 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 02:00:08.861231 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:08.860425 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vgz69" podUID="1941492f-5a14-4095-8743-70cb9e39ebe1" Jul 09 02:00:08.861231 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:08.860484 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trtrf" podUID="c5e07105-f26f-4ea9-8cde-9db06c456edc" Jul 09 02:00:09.104685 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:09.104656 2582 generic.go:356] "Generic (PLEG): container finished" podID="db97ff5f-15c4-4cef-b16c-fa03e39df587" containerID="1e5c363ce1a2333af23b0a928fd186edbf176ec2316a21267d3048b42d977c4b" exitCode=0 Jul 09 02:00:09.104916 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:09.104742 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hktj" event={"ID":"db97ff5f-15c4-4cef-b16c-fa03e39df587","Type":"ContainerDied","Data":"1e5c363ce1a2333af23b0a928fd186edbf176ec2316a21267d3048b42d977c4b"} Jul 09 02:00:09.108213 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:09.108197 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:00:09.108571 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:09.108546 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" event={"ID":"ec9d7c46-2a09-420b-90ae-5aad678928b0","Type":"ContainerStarted","Data":"07bd4bbc18775ddc611abc2508e1a3894775cf4017acec435aa640f9a42fd3dd"} Jul 09 02:00:09.108992 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:09.108980 2582 scope.go:117] "RemoveContainer" containerID="a6b6c328f8aed6829cb6f7037f8de43fbf31a1b3c654560cd9f02fbb994b03c9" Jul 09 02:00:09.861046 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:09.861013 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 02:00:09.861446 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:09.861139 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 02:00:10.116589 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:10.116159 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:00:10.116589 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:10.116557 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" event={"ID":"ec9d7c46-2a09-420b-90ae-5aad678928b0","Type":"ContainerStarted","Data":"62ad6cad4b5b363e9d9038989422d377d3b1c5086413310bcfaf34e7b43a02c0"} Jul 09 02:00:10.116715 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:10.116648 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 02:00:10.117202 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:10.116834 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 02:00:10.117202 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:10.116930 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 02:00:10.132586 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:10.132565 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 02:00:10.133072 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:10.133054 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 02:00:10.152660 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:10.152612 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" podStartSLOduration=9.261537421 podStartE2EDuration="30.152594499s" totalImagesPullingTime="20.891057078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 01:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:10.150318346 +0000 UTC m=+30.848372162" watchObservedRunningTime="2026-07-09 02:00:10.152594499 +0000 UTC m=+30.850648312" Jul 09 02:00:10.599124 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:10.598846 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vgz69"] Jul 09 02:00:10.599124 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:10.599020 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 02:00:10.599352 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:10.599130 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vgz69" podUID="1941492f-5a14-4095-8743-70cb9e39ebe1" Jul 09 02:00:10.599695 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:10.599596 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lg95f"] Jul 09 02:00:10.599987 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:10.599699 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 02:00:10.599987 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:10.599798 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 02:00:10.600164 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:10.600141 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-trtrf"] Jul 09 02:00:10.600247 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:10.600234 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 02:00:10.600336 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:10.600318 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trtrf" podUID="c5e07105-f26f-4ea9-8cde-9db06c456edc" Jul 09 02:00:10.832563 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:10.832541 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:10.933395 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:10.933367 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:11.121413 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:11.121332 2582 generic.go:356] "Generic (PLEG): container finished" podID="db97ff5f-15c4-4cef-b16c-fa03e39df587" containerID="001d8f34d9ab2daa6a8ba6dc796c23f19a95c79c5947cadabfa3e511ed0f51ae" exitCode=0 Jul 09 02:00:11.121572 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:11.121436 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hktj" event={"ID":"db97ff5f-15c4-4cef-b16c-fa03e39df587","Type":"ContainerDied","Data":"001d8f34d9ab2daa6a8ba6dc796c23f19a95c79c5947cadabfa3e511ed0f51ae"} Jul 09 02:00:11.859393 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:11.859362 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 02:00:11.859595 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:11.859466 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vgz69" podUID="1941492f-5a14-4095-8743-70cb9e39ebe1" Jul 09 02:00:11.859595 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:11.859514 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 02:00:11.859724 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:11.859614 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 02:00:12.539468 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:12.539234 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs\") pod \"network-metrics-daemon-trtrf\" (UID: \"c5e07105-f26f-4ea9-8cde-9db06c456edc\") " pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 02:00:12.539807 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:12.539512 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret\") pod \"global-pull-secret-syncer-lg95f\" (UID: \"7d15267e-8d0e-49e2-9a69-9c85f84b645f\") " pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 02:00:12.539807 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:12.539356 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 02:00:12.539807 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:12.539621 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs podName:c5e07105-f26f-4ea9-8cde-9db06c456edc nodeName:}" failed. No retries permitted until 2026-07-09 02:00:44.539598488 +0000 UTC m=+65.237652279 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs") pod "network-metrics-daemon-trtrf" (UID: "c5e07105-f26f-4ea9-8cde-9db06c456edc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 02:00:12.539807 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:12.539632 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 02:00:12.539807 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:12.539696 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret podName:7d15267e-8d0e-49e2-9a69-9c85f84b645f nodeName:}" failed. No retries permitted until 2026-07-09 02:00:44.539678518 +0000 UTC m=+65.237732313 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret") pod "global-pull-secret-syncer-lg95f" (UID: "7d15267e-8d0e-49e2-9a69-9c85f84b645f") : object "kube-system"/"original-pull-secret" not registered Jul 09 02:00:12.640857 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:12.640799 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmh2k\" (UniqueName: \"kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k\") pod \"network-check-target-vgz69\" (UID: \"1941492f-5a14-4095-8743-70cb9e39ebe1\") " pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 02:00:12.641026 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:12.640948 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 02:00:12.641026 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:12.640971 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 02:00:12.641026 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:12.640986 2582 projected.go:196] Error preparing data for projected volume kube-api-access-wmh2k for pod openshift-network-diagnostics/network-check-target-vgz69: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 02:00:12.641155 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:12.641034 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k podName:1941492f-5a14-4095-8743-70cb9e39ebe1 nodeName:}" failed. No retries permitted until 2026-07-09 02:00:44.641020632 +0000 UTC m=+65.339074423 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wmh2k" (UniqueName: "kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k") pod "network-check-target-vgz69" (UID: "1941492f-5a14-4095-8743-70cb9e39ebe1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 02:00:12.713021 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:12.712991 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:12.775956 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:12.775895 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:12.859819 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:12.859793 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 02:00:12.859937 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:12.859909 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trtrf" podUID="c5e07105-f26f-4ea9-8cde-9db06c456edc" Jul 09 02:00:13.128022 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:13.127943 2582 generic.go:356] "Generic (PLEG): container finished" podID="db97ff5f-15c4-4cef-b16c-fa03e39df587" containerID="9ad0cc28dc483db6af61f1c99087d2bf1ac9759f66ec923f9b69ed1c05bef6af" exitCode=0 Jul 09 02:00:13.128022 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:13.128003 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hktj" event={"ID":"db97ff5f-15c4-4cef-b16c-fa03e39df587","Type":"ContainerDied","Data":"9ad0cc28dc483db6af61f1c99087d2bf1ac9759f66ec923f9b69ed1c05bef6af"} Jul 09 02:00:13.859999 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:13.859955 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 02:00:13.860449 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:13.860071 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vgz69" podUID="1941492f-5a14-4095-8743-70cb9e39ebe1" Jul 09 02:00:13.860449 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:13.860129 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 02:00:13.860449 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:13.860217 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lg95f" podUID="7d15267e-8d0e-49e2-9a69-9c85f84b645f" Jul 09 02:00:13.934139 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:13.934105 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:14.035161 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.035121 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:14.062416 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.062393 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bb6hq_2a403528-9b59-4685-974e-69e52cac4ec4/dns-node-resolver/0.log" Jul 09 02:00:14.569081 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.569013 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-184.ec2.internal" event="NodeReady" Jul 09 02:00:14.569239 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.569140 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 02:00:14.629792 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.629760 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ckv5l"] Jul 09 02:00:14.631763 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.631740 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2btw2"] Jul 09 02:00:14.631923 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.631896 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ckv5l" Jul 09 02:00:14.633775 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.633758 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:14.635452 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.635430 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dnxq9\"" Jul 09 02:00:14.635786 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.635746 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 02:00:14.635786 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.635750 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 02:00:14.635786 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.635766 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 02:00:14.638464 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.638412 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 02:00:14.638571 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.638412 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8nxtt\"" Jul 09 02:00:14.638571 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.638417 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 02:00:14.649206 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.647845 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2btw2"] Jul 09 02:00:14.649206 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.648603 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ckv5l"] Jul 09 02:00:14.656085 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.656065 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wrbh6"] Jul 09 02:00:14.658901 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.658862 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:14.663068 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.662789 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 02:00:14.663068 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.662977 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vfnrl\"" Jul 09 02:00:14.663068 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.663041 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 02:00:14.663255 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.663169 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 02:00:14.663255 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.663216 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 02:00:14.677452 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.677427 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wrbh6"] Jul 09 02:00:14.757370 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.757340 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:14.757556 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.757378 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-crio-socket\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:14.757556 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.757463 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a081582-6289-42d1-aa82-f7faea30f410-metrics-tls\") pod \"dns-default-2btw2\" (UID: \"8a081582-6289-42d1-aa82-f7faea30f410\") " pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:14.757556 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.757497 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91ecf239-3824-49c2-aaab-26dc1679fde1-cert\") pod \"ingress-canary-ckv5l\" (UID: \"91ecf239-3824-49c2-aaab-26dc1679fde1\") " pod="openshift-ingress-canary/ingress-canary-ckv5l" Jul 09 02:00:14.757556 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.757531 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxr7p\" (UniqueName: \"kubernetes.io/projected/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-kube-api-access-wxr7p\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:14.757556 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.757552 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lkbj\" (UniqueName: \"kubernetes.io/projected/91ecf239-3824-49c2-aaab-26dc1679fde1-kube-api-access-8lkbj\") pod \"ingress-canary-ckv5l\" (UID: \"91ecf239-3824-49c2-aaab-26dc1679fde1\") " pod="openshift-ingress-canary/ingress-canary-ckv5l" Jul 09 02:00:14.757812 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.757575 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8a081582-6289-42d1-aa82-f7faea30f410-config-volume\") pod \"dns-default-2btw2\" (UID: \"8a081582-6289-42d1-aa82-f7faea30f410\") " pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:14.757812 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.757693 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-data-volume\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:14.757812 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.757739 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a081582-6289-42d1-aa82-f7faea30f410-tmp-dir\") pod \"dns-default-2btw2\" (UID: \"8a081582-6289-42d1-aa82-f7faea30f410\") " pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:14.757812 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.757767 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:14.758001 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.757827 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hgs8\" (UniqueName: \"kubernetes.io/projected/8a081582-6289-42d1-aa82-f7faea30f410-kube-api-access-5hgs8\") pod \"dns-default-2btw2\" (UID: \"8a081582-6289-42d1-aa82-f7faea30f410\") " pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:14.858557 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.858473 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a081582-6289-42d1-aa82-f7faea30f410-metrics-tls\") pod \"dns-default-2btw2\" (UID: \"8a081582-6289-42d1-aa82-f7faea30f410\") " pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:14.858557 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.858517 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91ecf239-3824-49c2-aaab-26dc1679fde1-cert\") pod \"ingress-canary-ckv5l\" (UID: \"91ecf239-3824-49c2-aaab-26dc1679fde1\") " pod="openshift-ingress-canary/ingress-canary-ckv5l" Jul 09 02:00:14.858557 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.858543 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxr7p\" (UniqueName: \"kubernetes.io/projected/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-kube-api-access-wxr7p\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:14.858820 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.858571 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lkbj\" (UniqueName: \"kubernetes.io/projected/91ecf239-3824-49c2-aaab-26dc1679fde1-kube-api-access-8lkbj\") pod \"ingress-canary-ckv5l\" (UID: \"91ecf239-3824-49c2-aaab-26dc1679fde1\") " pod="openshift-ingress-canary/ingress-canary-ckv5l" Jul 09 02:00:14.858820 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:14.858644 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 02:00:14.858820 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.858686 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8a081582-6289-42d1-aa82-f7faea30f410-config-volume\") pod \"dns-default-2btw2\" (UID: \"8a081582-6289-42d1-aa82-f7faea30f410\") " pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:14.858820 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:14.858721 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a081582-6289-42d1-aa82-f7faea30f410-metrics-tls podName:8a081582-6289-42d1-aa82-f7faea30f410 nodeName:}" failed. No retries permitted until 2026-07-09 02:00:15.358698485 +0000 UTC m=+36.056752279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8a081582-6289-42d1-aa82-f7faea30f410-metrics-tls") pod "dns-default-2btw2" (UID: "8a081582-6289-42d1-aa82-f7faea30f410") : secret "dns-default-metrics-tls" not found Jul 09 02:00:14.858820 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.858776 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-data-volume\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:14.858820 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.858812 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a081582-6289-42d1-aa82-f7faea30f410-tmp-dir\") pod \"dns-default-2btw2\" (UID: \"8a081582-6289-42d1-aa82-f7faea30f410\") " pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:14.859131 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.858839 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:14.859131 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.858895 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hgs8\" (UniqueName: \"kubernetes.io/projected/8a081582-6289-42d1-aa82-f7faea30f410-kube-api-access-5hgs8\") pod \"dns-default-2btw2\" (UID: \"8a081582-6289-42d1-aa82-f7faea30f410\") " pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:14.859131 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.858928 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:14.859131 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.858951 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-crio-socket\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:14.859131 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:14.859003 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 02:00:14.859131 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:14.859067 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-kube-rbac-proxy-cm podName:8495fb53-5899-4eb6-a4a1-9eca3914cdd1 nodeName:}" failed. No retries permitted until 2026-07-09 02:00:15.3590487 +0000 UTC m=+36.057102506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wrbh6" (UID: "8495fb53-5899-4eb6-a4a1-9eca3914cdd1") : configmap "kube-rbac-proxy" not found Jul 09 02:00:14.859131 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.859093 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-data-volume\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:14.859446 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.859175 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-crio-socket\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:14.859446 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:14.859194 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 02:00:14.859446 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.859235 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a081582-6289-42d1-aa82-f7faea30f410-tmp-dir\") pod \"dns-default-2btw2\" (UID: \"8a081582-6289-42d1-aa82-f7faea30f410\") " pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:14.859446 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:14.859253 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-insights-runtime-extractor-tls podName:8495fb53-5899-4eb6-a4a1-9eca3914cdd1 nodeName:}" failed. No retries permitted until 2026-07-09 02:00:15.359237319 +0000 UTC m=+36.057291134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wrbh6" (UID: "8495fb53-5899-4eb6-a4a1-9eca3914cdd1") : secret "insights-runtime-extractor-tls" not found Jul 09 02:00:14.859446 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.859279 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8a081582-6289-42d1-aa82-f7faea30f410-config-volume\") pod \"dns-default-2btw2\" (UID: \"8a081582-6289-42d1-aa82-f7faea30f410\") " pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:14.859446 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.859305 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 02:00:14.863647 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.863624 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 02:00:14.863647 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.863642 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pmqn4\"" Jul 09 02:00:14.864168 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.863920 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91ecf239-3824-49c2-aaab-26dc1679fde1-cert\") pod \"ingress-canary-ckv5l\" (UID: \"91ecf239-3824-49c2-aaab-26dc1679fde1\") " pod="openshift-ingress-canary/ingress-canary-ckv5l" Jul 09 02:00:14.871068 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.871019 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxr7p\" (UniqueName: \"kubernetes.io/projected/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-kube-api-access-wxr7p\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:14.871178 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.871068 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hgs8\" (UniqueName: \"kubernetes.io/projected/8a081582-6289-42d1-aa82-f7faea30f410-kube-api-access-5hgs8\") pod \"dns-default-2btw2\" (UID: \"8a081582-6289-42d1-aa82-f7faea30f410\") " pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:14.871491 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.871467 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lkbj\" (UniqueName: \"kubernetes.io/projected/91ecf239-3824-49c2-aaab-26dc1679fde1-kube-api-access-8lkbj\") pod \"ingress-canary-ckv5l\" (UID: \"91ecf239-3824-49c2-aaab-26dc1679fde1\") " pod="openshift-ingress-canary/ingress-canary-ckv5l" Jul 09 02:00:14.944137 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.944110 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ckv5l" Jul 09 02:00:14.984394 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:14.984363 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:15.047032 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:15.046998 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:15.063033 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:15.063010 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qkgn2_83da97a5-3688-4c0d-97b6-cc3cc3ed3d25/node-ca/0.log" Jul 09 02:00:15.092692 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:15.092662 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ckv5l"] Jul 09 02:00:15.097152 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:00:15.097127 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91ecf239_3824_49c2_aaab_26dc1679fde1.slice/crio-1d47cb36c827a930a393b049c2fa9a9bcac3c7997e27bcec52cbca74d08701a7 WatchSource:0}: Error finding container 1d47cb36c827a930a393b049c2fa9a9bcac3c7997e27bcec52cbca74d08701a7: Status 404 returned error can't find the container with id 1d47cb36c827a930a393b049c2fa9a9bcac3c7997e27bcec52cbca74d08701a7 Jul 09 02:00:15.113603 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:15.113545 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:15.134504 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:15.134473 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ckv5l" event={"ID":"91ecf239-3824-49c2-aaab-26dc1679fde1","Type":"ContainerStarted","Data":"1d47cb36c827a930a393b049c2fa9a9bcac3c7997e27bcec52cbca74d08701a7"} Jul 09 02:00:15.362456 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:15.362422 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:15.362653 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:15.362479 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:15.362653 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:15.362580 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 02:00:15.362653 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:15.362606 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a081582-6289-42d1-aa82-f7faea30f410-metrics-tls\") pod \"dns-default-2btw2\" (UID: \"8a081582-6289-42d1-aa82-f7faea30f410\") " pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:15.362653 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:15.362647 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-kube-rbac-proxy-cm podName:8495fb53-5899-4eb6-a4a1-9eca3914cdd1 nodeName:}" failed. No retries permitted until 2026-07-09 02:00:16.362630491 +0000 UTC m=+37.060684283 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wrbh6" (UID: "8495fb53-5899-4eb6-a4a1-9eca3914cdd1") : configmap "kube-rbac-proxy" not found Jul 09 02:00:15.362854 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:15.362710 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 02:00:15.362854 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:15.362735 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 02:00:15.362854 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:15.362757 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a081582-6289-42d1-aa82-f7faea30f410-metrics-tls podName:8a081582-6289-42d1-aa82-f7faea30f410 nodeName:}" failed. No retries permitted until 2026-07-09 02:00:16.362742958 +0000 UTC m=+37.060796756 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8a081582-6289-42d1-aa82-f7faea30f410-metrics-tls") pod "dns-default-2btw2" (UID: "8a081582-6289-42d1-aa82-f7faea30f410") : secret "dns-default-metrics-tls" not found Jul 09 02:00:15.362854 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:15.362786 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-insights-runtime-extractor-tls podName:8495fb53-5899-4eb6-a4a1-9eca3914cdd1 nodeName:}" failed. No retries permitted until 2026-07-09 02:00:16.362770722 +0000 UTC m=+37.060824527 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wrbh6" (UID: "8495fb53-5899-4eb6-a4a1-9eca3914cdd1") : secret "insights-runtime-extractor-tls" not found Jul 09 02:00:15.860182 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:15.859902 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 02:00:15.860761 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:15.859953 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 02:00:15.864185 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:15.864160 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j65lj\"" Jul 09 02:00:15.865788 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:15.865603 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 02:00:15.865788 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:15.865645 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 02:00:15.865982 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:15.865958 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 02:00:16.331220 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:16.331112 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:16.371454 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:16.371420 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a081582-6289-42d1-aa82-f7faea30f410-metrics-tls\") pod \"dns-default-2btw2\" (UID: \"8a081582-6289-42d1-aa82-f7faea30f410\") " pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:16.371646 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:16.371511 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:16.371646 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:16.371582 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:16.371646 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:16.371612 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 02:00:16.371803 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:16.371612 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 02:00:16.371803 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:16.371696 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-kube-rbac-proxy-cm podName:8495fb53-5899-4eb6-a4a1-9eca3914cdd1 nodeName:}" failed. No retries permitted until 2026-07-09 02:00:18.371674937 +0000 UTC m=+39.069728748 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wrbh6" (UID: "8495fb53-5899-4eb6-a4a1-9eca3914cdd1") : configmap "kube-rbac-proxy" not found Jul 09 02:00:16.371803 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:16.371712 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 02:00:16.371803 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:16.371730 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a081582-6289-42d1-aa82-f7faea30f410-metrics-tls podName:8a081582-6289-42d1-aa82-f7faea30f410 nodeName:}" failed. No retries permitted until 2026-07-09 02:00:18.371719319 +0000 UTC m=+39.069773112 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8a081582-6289-42d1-aa82-f7faea30f410-metrics-tls") pod "dns-default-2btw2" (UID: "8a081582-6289-42d1-aa82-f7faea30f410") : secret "dns-default-metrics-tls" not found Jul 09 02:00:16.371803 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:16.371764 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-insights-runtime-extractor-tls podName:8495fb53-5899-4eb6-a4a1-9eca3914cdd1 nodeName:}" failed. No retries permitted until 2026-07-09 02:00:18.37174873 +0000 UTC m=+39.069802534 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wrbh6" (UID: "8495fb53-5899-4eb6-a4a1-9eca3914cdd1") : secret "insights-runtime-extractor-tls" not found Jul 09 02:00:16.432338 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:16.432310 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:16.805931 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:16.805849 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:16.880975 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:16.880940 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:17.403289 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.403264 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:17.444324 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.444299 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-24dfr"] Jul 09 02:00:17.446404 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.446369 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-24dfr" Jul 09 02:00:17.449936 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.449918 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-2wzxj\"" Jul 09 02:00:17.451634 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.451610 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 09 02:00:17.451634 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.451628 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 09 02:00:17.461310 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.461288 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-24dfr"] Jul 09 02:00:17.500851 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.500811 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:17.582092 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.582058 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd8nw\" (UniqueName: \"kubernetes.io/projected/3f541a57-ab39-4408-8578-3bf6d9d1f8f0-kube-api-access-fd8nw\") pod \"migrator-d57d58cc9-24dfr\" (UID: \"3f541a57-ab39-4408-8578-3bf6d9d1f8f0\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-24dfr" Jul 09 02:00:17.598440 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.598412 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:17.632283 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.632253 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-l8gb2"] Jul 09 02:00:17.638804 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.638780 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l8gb2" Jul 09 02:00:17.642193 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.642172 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-klgn6\"" Jul 09 02:00:17.644360 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.644308 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-l8gb2"] Jul 09 02:00:17.668339 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.668277 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:17.682762 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.682737 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd8nw\" (UniqueName: \"kubernetes.io/projected/3f541a57-ab39-4408-8578-3bf6d9d1f8f0-kube-api-access-fd8nw\") pod \"migrator-d57d58cc9-24dfr\" (UID: \"3f541a57-ab39-4408-8578-3bf6d9d1f8f0\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-24dfr" Jul 09 02:00:17.696582 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.696552 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd8nw\" (UniqueName: \"kubernetes.io/projected/3f541a57-ab39-4408-8578-3bf6d9d1f8f0-kube-api-access-fd8nw\") pod \"migrator-d57d58cc9-24dfr\" (UID: \"3f541a57-ab39-4408-8578-3bf6d9d1f8f0\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-24dfr" Jul 09 02:00:17.756518 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.756490 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-24dfr" Jul 09 02:00:17.783957 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.783922 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw9sv\" (UniqueName: \"kubernetes.io/projected/0e2f0937-c209-4fe3-8715-6c36e4c6122e-kube-api-access-tw9sv\") pod \"network-check-source-989f8b8dc-l8gb2\" (UID: \"0e2f0937-c209-4fe3-8715-6c36e4c6122e\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l8gb2" Jul 09 02:00:17.815951 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.815910 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:17.885046 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.885011 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw9sv\" (UniqueName: \"kubernetes.io/projected/0e2f0937-c209-4fe3-8715-6c36e4c6122e-kube-api-access-tw9sv\") pod \"network-check-source-989f8b8dc-l8gb2\" (UID: \"0e2f0937-c209-4fe3-8715-6c36e4c6122e\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l8gb2" Jul 09 02:00:17.894622 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.894592 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw9sv\" (UniqueName: \"kubernetes.io/projected/0e2f0937-c209-4fe3-8715-6c36e4c6122e-kube-api-access-tw9sv\") pod \"network-check-source-989f8b8dc-l8gb2\" (UID: \"0e2f0937-c209-4fe3-8715-6c36e4c6122e\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l8gb2" Jul 09 02:00:17.949482 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:17.949402 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l8gb2" Jul 09 02:00:18.285668 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:18.285589 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:18.386094 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:18.386069 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:18.389087 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:18.389068 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:18.389175 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:18.389159 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 02:00:18.389239 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:18.389221 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:18.389270 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:18.389238 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-kube-rbac-proxy-cm podName:8495fb53-5899-4eb6-a4a1-9eca3914cdd1 nodeName:}" failed. No retries permitted until 2026-07-09 02:00:22.389219911 +0000 UTC m=+43.087273715 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wrbh6" (UID: "8495fb53-5899-4eb6-a4a1-9eca3914cdd1") : configmap "kube-rbac-proxy" not found Jul 09 02:00:18.389313 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:18.389298 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a081582-6289-42d1-aa82-f7faea30f410-metrics-tls\") pod \"dns-default-2btw2\" (UID: \"8a081582-6289-42d1-aa82-f7faea30f410\") " pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:18.389346 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:18.389311 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 02:00:18.389378 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:18.389352 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-insights-runtime-extractor-tls podName:8495fb53-5899-4eb6-a4a1-9eca3914cdd1 nodeName:}" failed. No retries permitted until 2026-07-09 02:00:22.38933953 +0000 UTC m=+43.087393322 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wrbh6" (UID: "8495fb53-5899-4eb6-a4a1-9eca3914cdd1") : secret "insights-runtime-extractor-tls" not found Jul 09 02:00:18.389430 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:18.389412 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 02:00:18.389496 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:18.389485 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a081582-6289-42d1-aa82-f7faea30f410-metrics-tls podName:8a081582-6289-42d1-aa82-f7faea30f410 nodeName:}" failed. No retries permitted until 2026-07-09 02:00:22.389469682 +0000 UTC m=+43.087523477 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8a081582-6289-42d1-aa82-f7faea30f410-metrics-tls") pod "dns-default-2btw2" (UID: "8a081582-6289-42d1-aa82-f7faea30f410") : secret "dns-default-metrics-tls" not found Jul 09 02:00:18.604358 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:18.604335 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-24dfr"] Jul 09 02:00:18.607242 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:18.607220 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-l8gb2"] Jul 09 02:00:18.608936 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:00:18.608909 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f541a57_ab39_4408_8578_3bf6d9d1f8f0.slice/crio-ea0acd7497e7672000eaa7a7fe8bce8f6a007a14b44e0f00c46d2105ba3aa1d9 WatchSource:0}: Error finding container ea0acd7497e7672000eaa7a7fe8bce8f6a007a14b44e0f00c46d2105ba3aa1d9: Status 404 returned error can't find the container with id ea0acd7497e7672000eaa7a7fe8bce8f6a007a14b44e0f00c46d2105ba3aa1d9 Jul 09 02:00:18.610150 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:00:18.610128 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e2f0937_c209_4fe3_8715_6c36e4c6122e.slice/crio-069353ee050c14bbc06330612aa2a503eca11ea5bc8bf52adfab4ae55005ba48 WatchSource:0}: Error finding container 069353ee050c14bbc06330612aa2a503eca11ea5bc8bf52adfab4ae55005ba48: Status 404 returned error can't find the container with id 069353ee050c14bbc06330612aa2a503eca11ea5bc8bf52adfab4ae55005ba48 Jul 09 02:00:19.145589 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:19.145335 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ckv5l" event={"ID":"91ecf239-3824-49c2-aaab-26dc1679fde1","Type":"ContainerStarted","Data":"a3b07474d1a03a21f5f46eb7e98cfa23fc921bd81dd1e8a02db4a98ad53169d6"} Jul 09 02:00:19.146508 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:19.146484 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-24dfr" event={"ID":"3f541a57-ab39-4408-8578-3bf6d9d1f8f0","Type":"ContainerStarted","Data":"ea0acd7497e7672000eaa7a7fe8bce8f6a007a14b44e0f00c46d2105ba3aa1d9"} Jul 09 02:00:19.147504 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:19.147481 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l8gb2" event={"ID":"0e2f0937-c209-4fe3-8715-6c36e4c6122e","Type":"ContainerStarted","Data":"069353ee050c14bbc06330612aa2a503eca11ea5bc8bf52adfab4ae55005ba48"} Jul 09 02:00:19.149985 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:19.149965 2582 generic.go:356] "Generic (PLEG): container finished" podID="db97ff5f-15c4-4cef-b16c-fa03e39df587" containerID="331f0fba77649d93408d89c537eda64c650136429b4287cfba6a6b07ecf6e2c1" exitCode=0 Jul 09 02:00:19.150051 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:19.150003 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hktj" event={"ID":"db97ff5f-15c4-4cef-b16c-fa03e39df587","Type":"ContainerDied","Data":"331f0fba77649d93408d89c537eda64c650136429b4287cfba6a6b07ecf6e2c1"} Jul 09 02:00:19.165110 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:19.165062 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ckv5l" podStartSLOduration=1.7978370890000002 podStartE2EDuration="5.165044637s" totalImagesPullingTime="3.367207548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:00:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:19.16494361 +0000 UTC m=+39.862997425" watchObservedRunningTime="2026-07-09 02:00:19.165044637 +0000 UTC m=+39.863098451" Jul 09 02:00:19.836937 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:19.836574 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:19.940918 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:19.940884 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:20.002764 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:20.002729 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:20.155165 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:20.155082 2582 generic.go:356] "Generic (PLEG): container finished" podID="db97ff5f-15c4-4cef-b16c-fa03e39df587" containerID="54ec3a8c369a399232a01d1f8e419268f0b676efcfd7cfcaffcb7f1d875758f7" exitCode=0 Jul 09 02:00:20.155165 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:20.155154 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hktj" event={"ID":"db97ff5f-15c4-4cef-b16c-fa03e39df587","Type":"ContainerDied","Data":"54ec3a8c369a399232a01d1f8e419268f0b676efcfd7cfcaffcb7f1d875758f7"} Jul 09 02:00:20.851170 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:20.851099 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:21.159133 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:21.159100 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-24dfr" event={"ID":"3f541a57-ab39-4408-8578-3bf6d9d1f8f0","Type":"ContainerStarted","Data":"75b5ad8a2bb1e730dbf7342d5a80e851b6e029a8ea91af668d3509e4fcdc641e"} Jul 09 02:00:21.159480 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:21.159161 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-24dfr" event={"ID":"3f541a57-ab39-4408-8578-3bf6d9d1f8f0","Type":"ContainerStarted","Data":"624b28df933984a916483149c8959b12146e0f685f9f2eb75d4f6845e0c7992f"} Jul 09 02:00:21.162241 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:21.162222 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hktj" event={"ID":"db97ff5f-15c4-4cef-b16c-fa03e39df587","Type":"ContainerStarted","Data":"a83d0d982148c41f330ae50c0fbaf74a911249cd6d4baaab4e65ac27ceed75a2"} Jul 09 02:00:21.183490 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:21.183453 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-24dfr" podStartSLOduration=1.871819979 podStartE2EDuration="4.18343998s" totalImagesPullingTime="2.311620001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:00:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:21.182735497 +0000 UTC m=+41.880789311" watchObservedRunningTime="2026-07-09 02:00:21.18343998 +0000 UTC m=+41.881493784" Jul 09 02:00:21.212683 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:21.212630 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6hktj" podStartSLOduration=8.073079914 podStartE2EDuration="41.212612222s" totalImagesPullingTime="33.139532308s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 01:59:40 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:21.212315227 +0000 UTC m=+41.910369052" watchObservedRunningTime="2026-07-09 02:00:21.212612222 +0000 UTC m=+41.910666036" Jul 09 02:00:22.380938 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:22.380903 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:22.420533 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:22.420508 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a081582-6289-42d1-aa82-f7faea30f410-metrics-tls\") pod \"dns-default-2btw2\" (UID: \"8a081582-6289-42d1-aa82-f7faea30f410\") " pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:22.420639 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:22.420552 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:22.420685 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:22.420651 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 02:00:22.420715 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:22.420687 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:22.420715 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:22.420702 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a081582-6289-42d1-aa82-f7faea30f410-metrics-tls podName:8a081582-6289-42d1-aa82-f7faea30f410 nodeName:}" failed. No retries permitted until 2026-07-09 02:00:30.420688486 +0000 UTC m=+51.118742277 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8a081582-6289-42d1-aa82-f7faea30f410-metrics-tls") pod "dns-default-2btw2" (UID: "8a081582-6289-42d1-aa82-f7faea30f410") : secret "dns-default-metrics-tls" not found Jul 09 02:00:22.420787 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:22.420770 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 02:00:22.420817 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:22.420810 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-insights-runtime-extractor-tls podName:8495fb53-5899-4eb6-a4a1-9eca3914cdd1 nodeName:}" failed. No retries permitted until 2026-07-09 02:00:30.420800348 +0000 UTC m=+51.118854139 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wrbh6" (UID: "8495fb53-5899-4eb6-a4a1-9eca3914cdd1") : secret "insights-runtime-extractor-tls" not found Jul 09 02:00:22.421059 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:22.421045 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:23.169123 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:23.169083 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l8gb2" event={"ID":"0e2f0937-c209-4fe3-8715-6c36e4c6122e","Type":"ContainerStarted","Data":"2f801f2b7b191dce34a589f6c0a49b93106c156c85259c6dcc2d8573ecc31dbc"} Jul 09 02:00:23.189312 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:23.189269 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l8gb2" podStartSLOduration=2.046131278 podStartE2EDuration="6.189254295s" totalImagesPullingTime="4.143123017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:00:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:23.188346016 +0000 UTC m=+43.886399830" watchObservedRunningTime="2026-07-09 02:00:23.189254295 +0000 UTC m=+43.887308107" Jul 09 02:00:23.927062 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:23.927031 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:24.464837 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:24.464792 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:24.644819 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:24.644786 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:24.744834 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:24.744750 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:24.839099 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:24.839068 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:24.910771 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:24.910747 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:25.360590 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:25.360559 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:25.933965 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:25.933935 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:26.529943 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:26.529911 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:26.737864 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:26.737835 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:27.140371 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:27.140341 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:27.330763 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:27.330724 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:27.737028 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:27.737001 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:28.130553 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:28.130462 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:28.455503 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:28.455465 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:28.555696 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:28.555665 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:28.738923 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:28.738814 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:29.130962 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:29.130888 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:29.197054 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:29.197021 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:29.337816 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:29.337773 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:29.734938 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:29.734905 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:29.834701 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:29.834673 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:29.949208 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:29.949179 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:30.049273 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:30.049204 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:30.349495 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:30.349411 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:30.481655 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:30.481618 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:30.481848 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:30.481677 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a081582-6289-42d1-aa82-f7faea30f410-metrics-tls\") pod \"dns-default-2btw2\" (UID: \"8a081582-6289-42d1-aa82-f7faea30f410\") " pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:30.484648 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:30.484616 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a081582-6289-42d1-aa82-f7faea30f410-metrics-tls\") pod \"dns-default-2btw2\" (UID: \"8a081582-6289-42d1-aa82-f7faea30f410\") " pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:30.484754 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:30.484650 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8495fb53-5899-4eb6-a4a1-9eca3914cdd1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wrbh6\" (UID: \"8495fb53-5899-4eb6-a4a1-9eca3914cdd1\") " pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:30.552158 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:30.552136 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:30.571282 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:30.571258 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wrbh6" Jul 09 02:00:30.651809 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:30.651778 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:30.706296 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:30.706273 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2btw2"] Jul 09 02:00:30.746811 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:30.746787 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wrbh6"] Jul 09 02:00:30.749056 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:00:30.749019 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8495fb53_5899_4eb6_a4a1_9eca3914cdd1.slice/crio-b34f23d3bc0beb79bac36601ec321a5aa4d7ee929c892ea46daeeb9ed03aa6d7 WatchSource:0}: Error finding container b34f23d3bc0beb79bac36601ec321a5aa4d7ee929c892ea46daeeb9ed03aa6d7: Status 404 returned error can't find the container with id b34f23d3bc0beb79bac36601ec321a5aa4d7ee929c892ea46daeeb9ed03aa6d7 Jul 09 02:00:30.938531 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:30.938501 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:31.153252 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:31.153222 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:31.189584 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:31.189497 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2btw2" event={"ID":"8a081582-6289-42d1-aa82-f7faea30f410","Type":"ContainerStarted","Data":"52345c7b6d415998b24f397d2af621764818db1bd2e6175d8e84e51469c7fbe8"} Jul 09 02:00:31.190441 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:31.190418 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wrbh6" event={"ID":"8495fb53-5899-4eb6-a4a1-9eca3914cdd1","Type":"ContainerStarted","Data":"b34f23d3bc0beb79bac36601ec321a5aa4d7ee929c892ea46daeeb9ed03aa6d7"} Jul 09 02:00:31.534401 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:31.534320 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:31.739196 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:31.739081 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:31.949298 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:31.949245 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:32.537509 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:32.537475 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:32.932496 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:32.932466 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:33.154061 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:33.154022 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:33.528141 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:33.528105 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:33.734767 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:33.734721 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:33.937196 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:33.937141 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:34.792772 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:34.792697 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:34.933574 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:34.933533 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:35.131628 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:35.131527 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:35.369632 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:35.369568 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:35.470040 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:35.470014 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:35.665402 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:35.665369 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:35.855133 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:35.855099 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:36.046067 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:36.046038 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:36.148010 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:36.147986 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:36.207414 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:36.207383 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2btw2" event={"ID":"8a081582-6289-42d1-aa82-f7faea30f410","Type":"ContainerStarted","Data":"15be74d944af8f3f4589c0f0ca03e2f0483c390d5f9fff94eda76b0fd0c3fe62"} Jul 09 02:00:36.207583 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:36.207440 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2btw2" event={"ID":"8a081582-6289-42d1-aa82-f7faea30f410","Type":"ContainerStarted","Data":"7d0a6d16a48beb7b611a6ecd6e4933f241d78cd2338f20ad8e35a1b7a0d3ca96"} Jul 09 02:00:36.207583 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:36.207505 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:36.209012 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:36.208993 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wrbh6" event={"ID":"8495fb53-5899-4eb6-a4a1-9eca3914cdd1","Type":"ContainerStarted","Data":"a6f351535def819c1b8df712edd2071c120c4619435daaa84dfdd01510ff3d89"} Jul 09 02:00:36.209091 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:36.209034 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wrbh6" event={"ID":"8495fb53-5899-4eb6-a4a1-9eca3914cdd1","Type":"ContainerStarted","Data":"11d399d28ac9f5fb2c71c19f4871e2fee0febf6dc70cb6ed44fdebe06210bc69"} Jul 09 02:00:36.226971 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:36.226931 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2btw2" podStartSLOduration=17.675142802 podStartE2EDuration="22.226917341s" totalImagesPullingTime="4.551774539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:00:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:36.226477288 +0000 UTC m=+56.924531103" watchObservedRunningTime="2026-07-09 02:00:36.226917341 +0000 UTC m=+56.924971156" Jul 09 02:00:36.449298 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:36.449207 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:36.551786 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:36.551727 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:36.648368 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:36.648325 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:36.714795 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:36.714697 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:37.270007 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:37.269979 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:37.465738 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:37.465709 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:37.674964 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:37.674926 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:38.504654 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:38.504618 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:38.819475 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:38.819443 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:38.916381 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:38.916349 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:39.221389 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:39.221351 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wrbh6" event={"ID":"8495fb53-5899-4eb6-a4a1-9eca3914cdd1","Type":"ContainerStarted","Data":"d4e56bd3edddf27571924c794c0cce2f870af263acc2184468ffd11d888a10f2"} Jul 09 02:00:39.242643 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:39.242598 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wrbh6" podStartSLOduration=17.828903766 podStartE2EDuration="25.242583711s" totalImagesPullingTime="7.413679945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 02:00:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:39.242029278 +0000 UTC m=+59.940083103" watchObservedRunningTime="2026-07-09 02:00:39.242583711 +0000 UTC m=+59.940637524" Jul 09 02:00:42.133772 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:42.133733 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lj28w" Jul 09 02:00:43.336241 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:43.336212 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:43.437071 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:43.437038 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:43.536944 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:43.536918 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:43.814155 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:43.814128 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:44.216278 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.216247 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx"] Jul 09 02:00:44.219072 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.219055 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.222510 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.222490 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 09 02:00:44.224248 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.224225 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 09 02:00:44.224386 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.224226 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 09 02:00:44.224386 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.224225 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 09 02:00:44.224817 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.224793 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 09 02:00:44.224974 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.224953 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 09 02:00:44.225499 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.225310 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 09 02:00:44.236405 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.236377 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx"] Jul 09 02:00:44.284393 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.284366 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2c931338-fc54-4da8-93cb-34afc11016da-ca\") pod \"cluster-proxy-proxy-agent-7b9d4b7446-mvphx\" (UID: \"2c931338-fc54-4da8-93cb-34afc11016da\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.284393 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.284395 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c931338-fc54-4da8-93cb-34afc11016da-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b9d4b7446-mvphx\" (UID: \"2c931338-fc54-4da8-93cb-34afc11016da\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.284537 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.284412 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngfxs\" (UniqueName: \"kubernetes.io/projected/2c931338-fc54-4da8-93cb-34afc11016da-kube-api-access-ngfxs\") pod \"cluster-proxy-proxy-agent-7b9d4b7446-mvphx\" (UID: \"2c931338-fc54-4da8-93cb-34afc11016da\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.284537 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.284430 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c931338-fc54-4da8-93cb-34afc11016da-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b9d4b7446-mvphx\" (UID: \"2c931338-fc54-4da8-93cb-34afc11016da\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.284537 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.284454 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2c931338-fc54-4da8-93cb-34afc11016da-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b9d4b7446-mvphx\" (UID: \"2c931338-fc54-4da8-93cb-34afc11016da\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.284537 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.284518 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2c931338-fc54-4da8-93cb-34afc11016da-hub\") pod \"cluster-proxy-proxy-agent-7b9d4b7446-mvphx\" (UID: \"2c931338-fc54-4da8-93cb-34afc11016da\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.325200 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.325172 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-78cht"] Jul 09 02:00:44.327105 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.327087 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-78cht" Jul 09 02:00:44.330437 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.330418 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-gwsqk\"" Jul 09 02:00:44.330591 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.330566 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 09 02:00:44.344635 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.344589 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-78cht"] Jul 09 02:00:44.346636 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.346615 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:44.359809 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.359787 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-57566d484-zxrgq"] Jul 09 02:00:44.361682 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.361669 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.365536 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.365520 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 02:00:44.365617 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.365555 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 02:00:44.365987 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.365958 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 02:00:44.366522 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.366506 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hmfxw\"" Jul 09 02:00:44.375785 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.375767 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 02:00:44.384838 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.384817 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c931338-fc54-4da8-93cb-34afc11016da-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b9d4b7446-mvphx\" (UID: \"2c931338-fc54-4da8-93cb-34afc11016da\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.384944 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.384855 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2c931338-fc54-4da8-93cb-34afc11016da-ca\") pod \"cluster-proxy-proxy-agent-7b9d4b7446-mvphx\" (UID: \"2c931338-fc54-4da8-93cb-34afc11016da\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.384944 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.384896 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c931338-fc54-4da8-93cb-34afc11016da-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b9d4b7446-mvphx\" (UID: \"2c931338-fc54-4da8-93cb-34afc11016da\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.384944 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.384928 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2c931338-fc54-4da8-93cb-34afc11016da-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b9d4b7446-mvphx\" (UID: \"2c931338-fc54-4da8-93cb-34afc11016da\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.385112 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.384958 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-image-registry-private-configuration\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.385112 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.384984 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2c931338-fc54-4da8-93cb-34afc11016da-hub\") pod \"cluster-proxy-proxy-agent-7b9d4b7446-mvphx\" (UID: \"2c931338-fc54-4da8-93cb-34afc11016da\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.385336 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.385305 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-registry-tls\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.385498 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.385480 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-ca-trust-extracted\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.385607 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.385593 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngfxs\" (UniqueName: \"kubernetes.io/projected/2c931338-fc54-4da8-93cb-34afc11016da-kube-api-access-ngfxs\") pod \"cluster-proxy-proxy-agent-7b9d4b7446-mvphx\" (UID: \"2c931338-fc54-4da8-93cb-34afc11016da\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.385967 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.385679 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c931338-fc54-4da8-93cb-34afc11016da-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b9d4b7446-mvphx\" (UID: \"2c931338-fc54-4da8-93cb-34afc11016da\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.385967 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.385689 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rql6h\" (UniqueName: \"kubernetes.io/projected/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-kube-api-access-rql6h\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.385967 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.385740 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-bound-sa-token\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.385967 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.385807 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57566d484-zxrgq"] Jul 09 02:00:44.385967 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.385788 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-registry-certificates\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.385967 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.385843 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fa1c593a-8ea7-4290-8677-83e96fa7bd9a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-78cht\" (UID: \"fa1c593a-8ea7-4290-8677-83e96fa7bd9a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-78cht" Jul 09 02:00:44.385967 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.385945 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-installation-pull-secrets\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.386338 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.385999 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-trusted-ca\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.387864 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.387841 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2c931338-fc54-4da8-93cb-34afc11016da-ca\") pod \"cluster-proxy-proxy-agent-7b9d4b7446-mvphx\" (UID: \"2c931338-fc54-4da8-93cb-34afc11016da\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.387974 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.387944 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2c931338-fc54-4da8-93cb-34afc11016da-hub\") pod \"cluster-proxy-proxy-agent-7b9d4b7446-mvphx\" (UID: \"2c931338-fc54-4da8-93cb-34afc11016da\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.388034 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.388015 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2c931338-fc54-4da8-93cb-34afc11016da-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b9d4b7446-mvphx\" (UID: \"2c931338-fc54-4da8-93cb-34afc11016da\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.388234 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.388214 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c931338-fc54-4da8-93cb-34afc11016da-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b9d4b7446-mvphx\" (UID: \"2c931338-fc54-4da8-93cb-34afc11016da\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.400777 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.400752 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngfxs\" (UniqueName: \"kubernetes.io/projected/2c931338-fc54-4da8-93cb-34afc11016da-kube-api-access-ngfxs\") pod \"cluster-proxy-proxy-agent-7b9d4b7446-mvphx\" (UID: \"2c931338-fc54-4da8-93cb-34afc11016da\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.483265 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.483198 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:44.487291 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.487274 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-registry-tls\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.487368 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.487304 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-ca-trust-extracted\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.487368 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.487321 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rql6h\" (UniqueName: \"kubernetes.io/projected/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-kube-api-access-rql6h\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.487368 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.487340 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-bound-sa-token\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.487368 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.487367 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-registry-certificates\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.487546 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.487383 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fa1c593a-8ea7-4290-8677-83e96fa7bd9a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-78cht\" (UID: \"fa1c593a-8ea7-4290-8677-83e96fa7bd9a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-78cht" Jul 09 02:00:44.487546 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.487418 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-installation-pull-secrets\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.487546 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.487441 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-trusted-ca\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.487546 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.487495 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-image-registry-private-configuration\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.487739 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.487694 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-ca-trust-extracted\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.488259 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.488235 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-registry-certificates\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.488518 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.488496 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-trusted-ca\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.489830 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.489800 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-image-registry-private-configuration\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.489942 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.489910 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-installation-pull-secrets\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.490207 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.490190 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fa1c593a-8ea7-4290-8677-83e96fa7bd9a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-78cht\" (UID: \"fa1c593a-8ea7-4290-8677-83e96fa7bd9a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-78cht" Jul 09 02:00:44.490301 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.490283 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-registry-tls\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.510077 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.510059 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-bound-sa-token\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.510272 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.510255 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rql6h\" (UniqueName: \"kubernetes.io/projected/a1ddebc5-dddc-47b0-b213-c5f15172a8e2-kube-api-access-rql6h\") pod \"image-registry-57566d484-zxrgq\" (UID: \"a1ddebc5-dddc-47b0-b213-c5f15172a8e2\") " pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.537132 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.537113 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" Jul 09 02:00:44.588397 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.588364 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs\") pod \"network-metrics-daemon-trtrf\" (UID: \"c5e07105-f26f-4ea9-8cde-9db06c456edc\") " pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 02:00:44.588495 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.588473 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret\") pod \"global-pull-secret-syncer-lg95f\" (UID: \"7d15267e-8d0e-49e2-9a69-9c85f84b645f\") " pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 02:00:44.593028 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.592848 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 02:00:44.593028 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.592851 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 02:00:44.601261 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.601239 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5e07105-f26f-4ea9-8cde-9db06c456edc-metrics-certs\") pod \"network-metrics-daemon-trtrf\" (UID: \"c5e07105-f26f-4ea9-8cde-9db06c456edc\") " pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 02:00:44.601360 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.601319 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d15267e-8d0e-49e2-9a69-9c85f84b645f-original-pull-secret\") pod \"global-pull-secret-syncer-lg95f\" (UID: \"7d15267e-8d0e-49e2-9a69-9c85f84b645f\") " pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 02:00:44.606697 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.606675 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:44.635288 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.635263 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-78cht" Jul 09 02:00:44.671049 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.671019 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:44.683042 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.682584 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lg95f" Jul 09 02:00:44.689086 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.689059 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmh2k\" (UniqueName: \"kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k\") pod \"network-check-target-vgz69\" (UID: \"1941492f-5a14-4095-8743-70cb9e39ebe1\") " pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 02:00:44.693010 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.692986 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmh2k\" (UniqueName: \"kubernetes.io/projected/1941492f-5a14-4095-8743-70cb9e39ebe1-kube-api-access-wmh2k\") pod \"network-check-target-vgz69\" (UID: \"1941492f-5a14-4095-8743-70cb9e39ebe1\") " pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 02:00:44.727333 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.727300 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx"] Jul 09 02:00:44.755689 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.755656 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:44.790734 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.790697 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-78cht"] Jul 09 02:00:44.792045 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:00:44.792011 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa1c593a_8ea7_4290_8677_83e96fa7bd9a.slice/crio-552142c4839a72e722f75b309a9ae692b6b30f662b43f20373126265b00a75e1 WatchSource:0}: Error finding container 552142c4839a72e722f75b309a9ae692b6b30f662b43f20373126265b00a75e1: Status 404 returned error can't find the container with id 552142c4839a72e722f75b309a9ae692b6b30f662b43f20373126265b00a75e1 Jul 09 02:00:44.825348 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.825320 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57566d484-zxrgq"] Jul 09 02:00:44.828377 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:00:44.828358 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ddebc5_dddc_47b0_b213_c5f15172a8e2.slice/crio-0b5ccb5d89bc4b101348c4c26f1aa397ce965c51b3b324324945aa19435c1b3a WatchSource:0}: Error finding container 0b5ccb5d89bc4b101348c4c26f1aa397ce965c51b3b324324945aa19435c1b3a: Status 404 returned error can't find the container with id 0b5ccb5d89bc4b101348c4c26f1aa397ce965c51b3b324324945aa19435c1b3a Jul 09 02:00:44.837402 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.837382 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lg95f"] Jul 09 02:00:44.841453 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:00:44.841431 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d15267e_8d0e_49e2_9a69_9c85f84b645f.slice/crio-692ff8b94650811e3a08d293debf08d6618003be0ee403193d2f2c23db320072 WatchSource:0}: Error finding container 692ff8b94650811e3a08d293debf08d6618003be0ee403193d2f2c23db320072: Status 404 returned error can't find the container with id 692ff8b94650811e3a08d293debf08d6618003be0ee403193d2f2c23db320072 Jul 09 02:00:44.872967 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.872944 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:44.886310 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.886291 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pmqn4\"" Jul 09 02:00:44.893724 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.893708 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trtrf" Jul 09 02:00:44.974761 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.974730 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:44.978765 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.978747 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j65lj\"" Jul 09 02:00:44.986700 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:44.986632 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 02:00:45.021787 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:45.021756 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-trtrf"] Jul 09 02:00:45.025541 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:00:45.025511 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5e07105_f26f_4ea9_8cde_9db06c456edc.slice/crio-65bd59b985e8612b5ab79eec6e0642e3826b0e6516458c75cd49d0b8c1dc95eb WatchSource:0}: Error finding container 65bd59b985e8612b5ab79eec6e0642e3826b0e6516458c75cd49d0b8c1dc95eb: Status 404 returned error can't find the container with id 65bd59b985e8612b5ab79eec6e0642e3826b0e6516458c75cd49d0b8c1dc95eb Jul 09 02:00:45.112494 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:45.112473 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vgz69"] Jul 09 02:00:45.114698 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:00:45.114670 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1941492f_5a14_4095_8743_70cb9e39ebe1.slice/crio-abcae691d65cc1c63ab40e6bb456f414c21bb49ac4bdb988411043f7398faa2c WatchSource:0}: Error finding container abcae691d65cc1c63ab40e6bb456f414c21bb49ac4bdb988411043f7398faa2c: Status 404 returned error can't find the container with id abcae691d65cc1c63ab40e6bb456f414c21bb49ac4bdb988411043f7398faa2c Jul 09 02:00:45.244994 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:45.244898 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lg95f" event={"ID":"7d15267e-8d0e-49e2-9a69-9c85f84b645f","Type":"ContainerStarted","Data":"692ff8b94650811e3a08d293debf08d6618003be0ee403193d2f2c23db320072"} Jul 09 02:00:45.246406 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:45.246380 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57566d484-zxrgq" event={"ID":"a1ddebc5-dddc-47b0-b213-c5f15172a8e2","Type":"ContainerStarted","Data":"549bc9b52cdbe44a05ce8705d4e6393a47b65df530b2ad2dd528cb39b776cc7c"} Jul 09 02:00:45.246528 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:45.246421 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57566d484-zxrgq" event={"ID":"a1ddebc5-dddc-47b0-b213-c5f15172a8e2","Type":"ContainerStarted","Data":"0b5ccb5d89bc4b101348c4c26f1aa397ce965c51b3b324324945aa19435c1b3a"} Jul 09 02:00:45.246528 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:45.246466 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:00:45.247895 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:45.247853 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vgz69" event={"ID":"1941492f-5a14-4095-8743-70cb9e39ebe1","Type":"ContainerStarted","Data":"a038e8b39a056e3afe83cd9f6d4ecd9e19f3637f18dc47d43454ba8ee23c97fa"} Jul 09 02:00:45.248010 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:45.247907 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vgz69" event={"ID":"1941492f-5a14-4095-8743-70cb9e39ebe1","Type":"ContainerStarted","Data":"abcae691d65cc1c63ab40e6bb456f414c21bb49ac4bdb988411043f7398faa2c"} Jul 09 02:00:45.248010 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:45.247950 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 02:00:45.249092 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:45.249068 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-trtrf" event={"ID":"c5e07105-f26f-4ea9-8cde-9db06c456edc","Type":"ContainerStarted","Data":"65bd59b985e8612b5ab79eec6e0642e3826b0e6516458c75cd49d0b8c1dc95eb"} Jul 09 02:00:45.250187 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:45.250167 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-78cht" event={"ID":"fa1c593a-8ea7-4290-8677-83e96fa7bd9a","Type":"ContainerStarted","Data":"552142c4839a72e722f75b309a9ae692b6b30f662b43f20373126265b00a75e1"} Jul 09 02:00:45.251307 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:45.251287 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" event={"ID":"2c931338-fc54-4da8-93cb-34afc11016da","Type":"ContainerStarted","Data":"1942f5e5cd472eda7a61349ef95e66468c2738d48a29bc8cb71ef655e090a8d1"} Jul 09 02:00:45.277462 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:45.277411 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-57566d484-zxrgq" podStartSLOduration=1.2773948370000001 podStartE2EDuration="1.277394837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:00:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:45.275505158 +0000 UTC m=+65.973558972" watchObservedRunningTime="2026-07-09 02:00:45.277394837 +0000 UTC m=+65.975448650" Jul 09 02:00:45.304655 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:45.304601 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vgz69" podStartSLOduration=65.304585827 podStartE2EDuration="1m5.304585827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 01:59:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:45.304017769 +0000 UTC m=+66.002071582" watchObservedRunningTime="2026-07-09 02:00:45.304585827 +0000 UTC m=+66.002639631" Jul 09 02:00:46.215848 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:46.215812 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2btw2" Jul 09 02:00:48.110292 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:48.110238 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:48.210503 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:48.210467 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:48.265499 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:48.265441 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-trtrf" event={"ID":"c5e07105-f26f-4ea9-8cde-9db06c456edc","Type":"ContainerStarted","Data":"c584947965e20a863d633e57b3463626fbaa8a930b95b3fb51655186c5661adc"} Jul 09 02:00:48.265499 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:48.265501 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-trtrf" event={"ID":"c5e07105-f26f-4ea9-8cde-9db06c456edc","Type":"ContainerStarted","Data":"18adb7510176f77b0e96099bce5e491b18c252fce2b6a339dae228c930d4bcce"} Jul 09 02:00:48.267160 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:48.267131 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-78cht" event={"ID":"fa1c593a-8ea7-4290-8677-83e96fa7bd9a","Type":"ContainerStarted","Data":"dd72f07d5be972865029d871fcf1310fdb0435aff903b1f6b704f458c205f029"} Jul 09 02:00:48.267384 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:48.267364 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-78cht" Jul 09 02:00:48.274989 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:48.274963 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-78cht" Jul 09 02:00:48.286667 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:48.286632 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:48.287359 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:48.287317 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-trtrf" podStartSLOduration=65.780036047 podStartE2EDuration="1m8.287301814s" totalImagesPullingTime="2.507265767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 01:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:48.285249056 +0000 UTC m=+68.983302870" watchObservedRunningTime="2026-07-09 02:00:48.287301814 +0000 UTC m=+68.985355626" Jul 09 02:00:48.303651 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:48.303611 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-78cht" podStartSLOduration=1.565062079 podStartE2EDuration="4.303598184s" totalImagesPullingTime="2.738536105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:00:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:48.302699578 +0000 UTC m=+69.000753393" watchObservedRunningTime="2026-07-09 02:00:48.303598184 +0000 UTC m=+69.001651975" Jul 09 02:00:48.444655 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:48.444609 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:48.546466 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:48.546416 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:48.621437 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:48.621398 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:48.690994 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:48.690950 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:48.792466 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:48.792365 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:48.858845 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:48.858812 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:49.523592 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:49.523510 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:49.586738 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:49.586710 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:50.095314 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:50.095285 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:50.197159 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:50.197124 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:50.276726 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:50.276693 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lg95f" event={"ID":"7d15267e-8d0e-49e2-9a69-9c85f84b645f","Type":"ContainerStarted","Data":"34b7d44fd57d0062f567433e596fa7df8122ed27523ded106858c7b129e9a1a8"} Jul 09 02:00:50.302255 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:50.302200 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lg95f" podStartSLOduration=65.900652144 podStartE2EDuration="1m10.302185522s" totalImagesPullingTime="4.401533378s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 01:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:50.300362135 +0000 UTC m=+70.998415945" watchObservedRunningTime="2026-07-09 02:00:50.302185522 +0000 UTC m=+71.000239391" Jul 09 02:00:51.280936 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:51.280908 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" event={"ID":"2c931338-fc54-4da8-93cb-34afc11016da","Type":"ContainerStarted","Data":"2310b601a7ca77835252d8890de3d57f6b93d665621ff2399475fa240ecff663"} Jul 09 02:00:52.286041 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:52.286011 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" event={"ID":"2c931338-fc54-4da8-93cb-34afc11016da","Type":"ContainerStarted","Data":"e425e7d2f0ba95dfd13f018465fdccba5b4ad822f30e6399aefef8233382408e"} Jul 09 02:00:52.286457 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:52.286052 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" event={"ID":"2c931338-fc54-4da8-93cb-34afc11016da","Type":"ContainerStarted","Data":"5111233ec0ab80b84bd7fd6e4264290e5be2926a663ec32ff578f5d67c070c07"} Jul 09 02:00:52.310128 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:52.310078 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b9d4b7446-mvphx" podStartSLOduration=1.839748013 podStartE2EDuration="8.310064076s" totalImagesPullingTime="6.470316063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:00:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:00:52.307621913 +0000 UTC m=+73.005675726" watchObservedRunningTime="2026-07-09 02:00:52.310064076 +0000 UTC m=+73.008117889" Jul 09 02:00:53.694371 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:53.694331 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:53.793658 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:53.793627 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:55.299719 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:55.299692 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:55.399102 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:55.399070 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:55.500032 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:55.499996 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:57.751983 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:57.751952 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:57.909074 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:57.909041 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:58.010063 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.009977 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:58.094156 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.094129 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ssczb"] Jul 09 02:00:58.097759 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.097744 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.102576 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.102556 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 02:00:58.102981 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.102956 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 02:00:58.103073 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.103032 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 02:00:58.103586 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.103573 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 02:00:58.104434 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.104404 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 02:00:58.104524 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.104442 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zz76p\"" Jul 09 02:00:58.104904 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.104889 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 02:00:58.143060 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.143036 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:58.193907 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.193885 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3640bda0-e436-4577-8c36-6661763d217a-sys\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.194019 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.193932 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3640bda0-e436-4577-8c36-6661763d217a-metrics-client-ca\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.194019 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.193960 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3640bda0-e436-4577-8c36-6661763d217a-node-exporter-wtmp\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.194019 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.193979 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb6jg\" (UniqueName: \"kubernetes.io/projected/3640bda0-e436-4577-8c36-6661763d217a-kube-api-access-sb6jg\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.194019 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.194003 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3640bda0-e436-4577-8c36-6661763d217a-node-exporter-tls\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.194177 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.194032 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3640bda0-e436-4577-8c36-6661763d217a-node-exporter-textfile\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.194177 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.194053 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3640bda0-e436-4577-8c36-6661763d217a-node-exporter-accelerators-collector-config\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.194177 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.194125 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3640bda0-e436-4577-8c36-6661763d217a-root\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.194177 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.194151 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3640bda0-e436-4577-8c36-6661763d217a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.244245 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.244221 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:58.294737 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.294679 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3640bda0-e436-4577-8c36-6661763d217a-sys\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.294737 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.294709 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3640bda0-e436-4577-8c36-6661763d217a-metrics-client-ca\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.294737 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.294727 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3640bda0-e436-4577-8c36-6661763d217a-node-exporter-wtmp\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.294990 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.294741 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb6jg\" (UniqueName: \"kubernetes.io/projected/3640bda0-e436-4577-8c36-6661763d217a-kube-api-access-sb6jg\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.294990 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.294758 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3640bda0-e436-4577-8c36-6661763d217a-node-exporter-tls\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.294990 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.294783 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3640bda0-e436-4577-8c36-6661763d217a-node-exporter-textfile\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.294990 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.294807 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3640bda0-e436-4577-8c36-6661763d217a-sys\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.294990 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.294807 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3640bda0-e436-4577-8c36-6661763d217a-node-exporter-accelerators-collector-config\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.294990 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.294866 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3640bda0-e436-4577-8c36-6661763d217a-root\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.294990 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.294904 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3640bda0-e436-4577-8c36-6661763d217a-node-exporter-wtmp\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.294990 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:58.294911 2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 09 02:00:58.294990 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.294940 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3640bda0-e436-4577-8c36-6661763d217a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.294990 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.294926 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3640bda0-e436-4577-8c36-6661763d217a-root\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.294990 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:00:58.294996 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3640bda0-e436-4577-8c36-6661763d217a-node-exporter-tls podName:3640bda0-e436-4577-8c36-6661763d217a nodeName:}" failed. No retries permitted until 2026-07-09 02:00:58.794975602 +0000 UTC m=+79.493029412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3640bda0-e436-4577-8c36-6661763d217a-node-exporter-tls") pod "node-exporter-ssczb" (UID: "3640bda0-e436-4577-8c36-6661763d217a") : secret "node-exporter-tls" not found Jul 09 02:00:58.295550 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.295237 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3640bda0-e436-4577-8c36-6661763d217a-node-exporter-textfile\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.295550 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.295394 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3640bda0-e436-4577-8c36-6661763d217a-metrics-client-ca\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.295550 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.295450 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3640bda0-e436-4577-8c36-6661763d217a-node-exporter-accelerators-collector-config\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.297102 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.297083 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3640bda0-e436-4577-8c36-6661763d217a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.304361 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.304341 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb6jg\" (UniqueName: \"kubernetes.io/projected/3640bda0-e436-4577-8c36-6661763d217a-kube-api-access-sb6jg\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.347951 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.347927 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:58.447736 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.447710 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:58.631916 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.631853 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:58.735086 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.735053 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:58.799138 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.799109 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3640bda0-e436-4577-8c36-6661763d217a-node-exporter-tls\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.801265 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.801245 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3640bda0-e436-4577-8c36-6661763d217a-node-exporter-tls\") pod \"node-exporter-ssczb\" (UID: \"3640bda0-e436-4577-8c36-6661763d217a\") " pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:58.835243 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.835217 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:58.955694 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:58.955666 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:59.006949 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:59.006924 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ssczb" Jul 09 02:00:59.016827 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:00:59.016797 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3640bda0_e436_4577_8c36_6661763d217a.slice/crio-78ec3eb94c2ab83e0b7fcf3a7b0708e73f3895410b053f7de3101f78fb935be0 WatchSource:0}: Error finding container 78ec3eb94c2ab83e0b7fcf3a7b0708e73f3895410b053f7de3101f78fb935be0: Status 404 returned error can't find the container with id 78ec3eb94c2ab83e0b7fcf3a7b0708e73f3895410b053f7de3101f78fb935be0 Jul 09 02:00:59.062135 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:59.062103 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:59.169353 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:59.169323 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:59.310158 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:59.310078 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ssczb" event={"ID":"3640bda0-e436-4577-8c36-6661763d217a","Type":"ContainerStarted","Data":"78ec3eb94c2ab83e0b7fcf3a7b0708e73f3895410b053f7de3101f78fb935be0"} Jul 09 02:00:59.580589 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:59.580514 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:00:59.971227 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:00:59.971198 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:00.068141 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:00.068103 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:00.314950 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:00.314910 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ssczb" event={"ID":"3640bda0-e436-4577-8c36-6661763d217a","Type":"ContainerStarted","Data":"c7e6660c45830edb3013acbd2a8f348838b0d7372f69e4b598f7c7804ce5cab9"} Jul 09 02:01:01.040716 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:01.040584 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:01.140436 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:01.140408 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:01.242615 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:01.242586 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:01.320011 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:01.319911 2582 generic.go:356] "Generic (PLEG): container finished" podID="3640bda0-e436-4577-8c36-6661763d217a" containerID="c7e6660c45830edb3013acbd2a8f348838b0d7372f69e4b598f7c7804ce5cab9" exitCode=0 Jul 09 02:01:01.320011 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:01.319976 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ssczb" event={"ID":"3640bda0-e436-4577-8c36-6661763d217a","Type":"ContainerDied","Data":"c7e6660c45830edb3013acbd2a8f348838b0d7372f69e4b598f7c7804ce5cab9"} Jul 09 02:01:01.344003 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:01.343980 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:01.830436 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:01.830397 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:02.325163 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.325135 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ssczb" event={"ID":"3640bda0-e436-4577-8c36-6661763d217a","Type":"ContainerStarted","Data":"27ced1f056f1c7eaa442d98f4c9d8ccf3730184aa62f765c00143e81d7b90342"} Jul 09 02:01:02.325504 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.325181 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ssczb" event={"ID":"3640bda0-e436-4577-8c36-6661763d217a","Type":"ContainerStarted","Data":"69444b80556fcaaea37498f20916253bfd1a1506059e96e989fa5a1ba5f64d28"} Jul 09 02:01:02.479221 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.479171 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ssczb" podStartSLOduration=3.201323448 podStartE2EDuration="4.479153643s" totalImagesPullingTime="1.208848242s" totalInitContainerRuntime="68.981953ms" isStatefulPod=true podCreationTimestamp="2026-07-09 02:00:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:01:02.356799967 +0000 UTC m=+83.054853780" watchObservedRunningTime="2026-07-09 02:01:02.479153643 +0000 UTC m=+83.177207480" Jul 09 02:01:02.479377 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.479302 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-c4bd9748-29rrw"] Jul 09 02:01:02.482320 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.482305 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.487334 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.487298 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 09 02:01:02.487461 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.487386 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 02:01:02.487522 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.487506 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pwpw6\"" Jul 09 02:01:02.487617 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.487595 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 09 02:01:02.487843 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.487814 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 09 02:01:02.487843 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.487838 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-c1dfb9m12fu00\"" Jul 09 02:01:02.493484 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.493465 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-c4bd9748-29rrw"] Jul 09 02:01:02.630526 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.630456 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/90b74197-f326-4f3f-97b0-f30b99287cdd-metrics-server-audit-profiles\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.630526 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.630491 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/90b74197-f326-4f3f-97b0-f30b99287cdd-client-ca-bundle\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.630526 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.630516 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/90b74197-f326-4f3f-97b0-f30b99287cdd-audit-log\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.630805 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.630593 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90b74197-f326-4f3f-97b0-f30b99287cdd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.630805 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.630628 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb77q\" (UniqueName: \"kubernetes.io/projected/90b74197-f326-4f3f-97b0-f30b99287cdd-kube-api-access-sb77q\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.630805 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.630661 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/90b74197-f326-4f3f-97b0-f30b99287cdd-secret-metrics-server-tls\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.630805 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.630698 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/90b74197-f326-4f3f-97b0-f30b99287cdd-secret-metrics-server-client-certs\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.731397 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.731367 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/90b74197-f326-4f3f-97b0-f30b99287cdd-secret-metrics-server-client-certs\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.731553 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.731418 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/90b74197-f326-4f3f-97b0-f30b99287cdd-metrics-server-audit-profiles\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.731553 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.731442 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/90b74197-f326-4f3f-97b0-f30b99287cdd-client-ca-bundle\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.731553 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.731471 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/90b74197-f326-4f3f-97b0-f30b99287cdd-audit-log\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.731553 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.731504 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90b74197-f326-4f3f-97b0-f30b99287cdd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.731553 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.731528 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb77q\" (UniqueName: \"kubernetes.io/projected/90b74197-f326-4f3f-97b0-f30b99287cdd-kube-api-access-sb77q\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.731814 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.731659 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/90b74197-f326-4f3f-97b0-f30b99287cdd-secret-metrics-server-tls\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.732065 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.732033 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/90b74197-f326-4f3f-97b0-f30b99287cdd-audit-log\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.732230 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.732212 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90b74197-f326-4f3f-97b0-f30b99287cdd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.732581 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.732561 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/90b74197-f326-4f3f-97b0-f30b99287cdd-metrics-server-audit-profiles\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.733835 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.733817 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/90b74197-f326-4f3f-97b0-f30b99287cdd-client-ca-bundle\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.734024 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.734007 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/90b74197-f326-4f3f-97b0-f30b99287cdd-secret-metrics-server-client-certs\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.734087 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.734071 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/90b74197-f326-4f3f-97b0-f30b99287cdd-secret-metrics-server-tls\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.739993 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.739975 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb77q\" (UniqueName: \"kubernetes.io/projected/90b74197-f326-4f3f-97b0-f30b99287cdd-kube-api-access-sb77q\") pod \"metrics-server-c4bd9748-29rrw\" (UID: \"90b74197-f326-4f3f-97b0-f30b99287cdd\") " pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.745592 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.745566 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:02.790548 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.790522 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:02.832790 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.832755 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:02.918168 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:02.918146 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-c4bd9748-29rrw"] Jul 09 02:01:02.920412 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:01:02.920389 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90b74197_f326_4f3f_97b0_f30b99287cdd.slice/crio-caee6ee86d2aec87bf33dc026df6499a8c252aa9fb807a998de53f92049520cc WatchSource:0}: Error finding container caee6ee86d2aec87bf33dc026df6499a8c252aa9fb807a998de53f92049520cc: Status 404 returned error can't find the container with id caee6ee86d2aec87bf33dc026df6499a8c252aa9fb807a998de53f92049520cc Jul 09 02:01:03.203821 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:03.203741 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:03.333251 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:03.333213 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" event={"ID":"90b74197-f326-4f3f-97b0-f30b99287cdd","Type":"ContainerStarted","Data":"caee6ee86d2aec87bf33dc026df6499a8c252aa9fb807a998de53f92049520cc"} Jul 09 02:01:04.040766 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:04.040734 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:04.240682 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:04.240647 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:04.435724 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:04.435691 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:05.341706 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:05.341669 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" event={"ID":"90b74197-f326-4f3f-97b0-f30b99287cdd","Type":"ContainerStarted","Data":"c97e6530b59f6bfff5c9285f8dad42b295a97381c0c0e157106a258cc6faf630"} Jul 09 02:01:05.363697 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:05.363648 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" podStartSLOduration=1.026726286 podStartE2EDuration="3.363632064s" totalImagesPullingTime="2.336905778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:01:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:01:05.361724658 +0000 UTC m=+86.059778494" watchObservedRunningTime="2026-07-09 02:01:05.363632064 +0000 UTC m=+86.061685877" Jul 09 02:01:06.261134 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:06.261109 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-57566d484-zxrgq" Jul 09 02:01:06.397430 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:06.397397 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:06.501205 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:06.501174 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:06.600047 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:06.599976 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:07.431254 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:07.431219 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:07.534787 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:07.534743 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:07.634180 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:07.634148 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:09.154065 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:09.154024 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:09.254962 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:09.254921 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:13.448307 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:13.448277 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:13.549183 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:13.549154 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:14.169463 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:14.169426 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:14.283307 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:14.283276 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:14.938958 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:14.938923 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:15.039513 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:15.039468 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:16.259285 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:16.259259 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vgz69" Jul 09 02:01:22.790957 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:22.790921 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:22.791340 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:22.790982 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:29.939533 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:29.939502 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:30.001028 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:30.000992 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:30.935843 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:30.935811 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:01:42.796282 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:42.796250 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:01:42.800010 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:01:42.799992 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-c4bd9748-29rrw" Jul 09 02:02:17.493142 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:02:17.493112 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:02:17.874486 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:02:17.874408 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:02:27.112679 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:02:27.112643 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:02:27.195828 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:02:27.195803 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:02:37.453221 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:02:37.453146 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:02:37.554788 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:02:37.554758 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:02:37.616917 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:02:37.616890 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:03:32.792433 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:03:32.792398 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:03:32.865720 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:03:32.865694 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:03:35.642640 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:03:35.642609 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:03:35.743051 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:03:35.743017 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:03:35.844313 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:03:35.844277 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:03:36.425156 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:03:36.425127 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:03:36.524920 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:03:36.524885 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:03:44.768704 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:03:44.768670 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:03:44.827009 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:03:44.826981 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:03:46.727882 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:03:46.727846 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:03:46.840089 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:03:46.840057 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:03:46.900833 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:03:46.900801 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:03:59.940983 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:03:59.940951 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:04:00.040056 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:04:00.040026 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:04:00.095089 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:04:00.095063 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:04:12.416459 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:04:12.416385 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:04:12.475637 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:04:12.475610 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:04:38.140025 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:04:38.139980 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:04:39.484851 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:04:39.484819 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:04:39.686527 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:04:39.686503 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:04:39.689204 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:04:39.689177 2582 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 02:04:39.693502 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:04:39.693484 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:05:00.368736 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:00.368701 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:05.181726 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:05.181688 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:06.109663 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:06.109632 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:08.778501 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:08.778462 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:08.874423 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:08.874391 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:11.062358 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.062325 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:11.160456 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.160426 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:11.405994 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.405958 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:11.491599 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.491569 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:11.665744 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.665686 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh"] Jul 09 02:05:11.668393 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.668376 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh" Jul 09 02:05:11.671712 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.671693 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5m8gr\"" Jul 09 02:05:11.672041 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.672017 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 02:05:11.673202 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.673182 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 02:05:11.680377 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.680357 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh"] Jul 09 02:05:11.834550 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.834517 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9px4\" (UniqueName: \"kubernetes.io/projected/7e9a20e6-e100-44fe-bc33-77f399a0c262-kube-api-access-m9px4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh\" (UID: \"7e9a20e6-e100-44fe-bc33-77f399a0c262\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh" Jul 09 02:05:11.834706 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.834556 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e9a20e6-e100-44fe-bc33-77f399a0c262-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh\" (UID: \"7e9a20e6-e100-44fe-bc33-77f399a0c262\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh" Jul 09 02:05:11.834706 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.834634 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e9a20e6-e100-44fe-bc33-77f399a0c262-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh\" (UID: \"7e9a20e6-e100-44fe-bc33-77f399a0c262\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh" Jul 09 02:05:11.935496 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.935418 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e9a20e6-e100-44fe-bc33-77f399a0c262-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh\" (UID: \"7e9a20e6-e100-44fe-bc33-77f399a0c262\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh" Jul 09 02:05:11.935496 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.935490 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9px4\" (UniqueName: \"kubernetes.io/projected/7e9a20e6-e100-44fe-bc33-77f399a0c262-kube-api-access-m9px4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh\" (UID: \"7e9a20e6-e100-44fe-bc33-77f399a0c262\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh" Jul 09 02:05:11.935714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.935519 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e9a20e6-e100-44fe-bc33-77f399a0c262-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh\" (UID: \"7e9a20e6-e100-44fe-bc33-77f399a0c262\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh" Jul 09 02:05:11.935810 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.935778 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e9a20e6-e100-44fe-bc33-77f399a0c262-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh\" (UID: \"7e9a20e6-e100-44fe-bc33-77f399a0c262\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh" Jul 09 02:05:11.935907 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.935835 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e9a20e6-e100-44fe-bc33-77f399a0c262-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh\" (UID: \"7e9a20e6-e100-44fe-bc33-77f399a0c262\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh" Jul 09 02:05:11.946628 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.946599 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9px4\" (UniqueName: \"kubernetes.io/projected/7e9a20e6-e100-44fe-bc33-77f399a0c262-kube-api-access-m9px4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh\" (UID: \"7e9a20e6-e100-44fe-bc33-77f399a0c262\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh" Jul 09 02:05:11.977710 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:11.977689 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh" Jul 09 02:05:12.106591 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:12.106569 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh"] Jul 09 02:05:12.108614 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:05:12.108579 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e9a20e6_e100_44fe_bc33_77f399a0c262.slice/crio-d2ba574429864606585f04a8fc42a199a8d183b781265e3e2956bed7d1987fab WatchSource:0}: Error finding container d2ba574429864606585f04a8fc42a199a8d183b781265e3e2956bed7d1987fab: Status 404 returned error can't find the container with id d2ba574429864606585f04a8fc42a199a8d183b781265e3e2956bed7d1987fab Jul 09 02:05:12.110406 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:12.110390 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 02:05:12.140067 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:12.140034 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh" event={"ID":"7e9a20e6-e100-44fe-bc33-77f399a0c262","Type":"ContainerStarted","Data":"d2ba574429864606585f04a8fc42a199a8d183b781265e3e2956bed7d1987fab"} Jul 09 02:05:13.268334 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:13.268304 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:13.856114 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:13.856083 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-g7cml"] Jul 09 02:05:13.858410 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:13.858371 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-g7cml" Jul 09 02:05:13.861405 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:13.861383 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-qwmfp\"" Jul 09 02:05:13.862934 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:13.862908 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 09 02:05:13.863043 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:13.862942 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 09 02:05:13.866536 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:13.866515 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-g7cml"] Jul 09 02:05:13.886494 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:13.886468 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:13.951409 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:13.951379 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds7d7\" (UniqueName: \"kubernetes.io/projected/47491d80-01a4-4c93-870b-30839c14cd92-kube-api-access-ds7d7\") pod \"cert-manager-cainjector-559cdd8c87-g7cml\" (UID: \"47491d80-01a4-4c93-870b-30839c14cd92\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-g7cml" Jul 09 02:05:13.951516 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:13.951450 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47491d80-01a4-4c93-870b-30839c14cd92-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-g7cml\" (UID: \"47491d80-01a4-4c93-870b-30839c14cd92\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-g7cml" Jul 09 02:05:14.052720 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:14.052675 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47491d80-01a4-4c93-870b-30839c14cd92-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-g7cml\" (UID: \"47491d80-01a4-4c93-870b-30839c14cd92\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-g7cml" Jul 09 02:05:14.052905 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:14.052754 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds7d7\" (UniqueName: \"kubernetes.io/projected/47491d80-01a4-4c93-870b-30839c14cd92-kube-api-access-ds7d7\") pod \"cert-manager-cainjector-559cdd8c87-g7cml\" (UID: \"47491d80-01a4-4c93-870b-30839c14cd92\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-g7cml" Jul 09 02:05:14.061943 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:14.061916 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47491d80-01a4-4c93-870b-30839c14cd92-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-g7cml\" (UID: \"47491d80-01a4-4c93-870b-30839c14cd92\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-g7cml" Jul 09 02:05:14.062230 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:14.062207 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds7d7\" (UniqueName: \"kubernetes.io/projected/47491d80-01a4-4c93-870b-30839c14cd92-kube-api-access-ds7d7\") pod \"cert-manager-cainjector-559cdd8c87-g7cml\" (UID: \"47491d80-01a4-4c93-870b-30839c14cd92\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-g7cml" Jul 09 02:05:14.170724 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:14.170695 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-g7cml" Jul 09 02:05:14.299773 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:14.299741 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-g7cml"] Jul 09 02:05:14.303025 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:05:14.303001 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47491d80_01a4_4c93_870b_30839c14cd92.slice/crio-3d67ad2bd5c1990c6c9f90dceb11c30aad7c32d9dcf7c543cdb97792ce40f788 WatchSource:0}: Error finding container 3d67ad2bd5c1990c6c9f90dceb11c30aad7c32d9dcf7c543cdb97792ce40f788: Status 404 returned error can't find the container with id 3d67ad2bd5c1990c6c9f90dceb11c30aad7c32d9dcf7c543cdb97792ce40f788 Jul 09 02:05:15.153560 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:15.153478 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-g7cml" event={"ID":"47491d80-01a4-4c93-870b-30839c14cd92","Type":"ContainerStarted","Data":"3d67ad2bd5c1990c6c9f90dceb11c30aad7c32d9dcf7c543cdb97792ce40f788"} Jul 09 02:05:15.953800 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:15.953720 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:16.015690 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:16.015658 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:20.174940 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:20.174908 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-g7cml" event={"ID":"47491d80-01a4-4c93-870b-30839c14cd92","Type":"ContainerStarted","Data":"b4d5bcf5809f5e1935b120efc15b5fa977c6904f38ae06bad53055cd1de2b40d"} Jul 09 02:05:20.176349 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:20.176325 2582 generic.go:356] "Generic (PLEG): container finished" podID="7e9a20e6-e100-44fe-bc33-77f399a0c262" containerID="14721880f98482c031a3e164fab2659daa09e397e9a0857656b0d41f5be2d5f3" exitCode=0 Jul 09 02:05:20.176461 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:20.176371 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh" event={"ID":"7e9a20e6-e100-44fe-bc33-77f399a0c262","Type":"ContainerDied","Data":"14721880f98482c031a3e164fab2659daa09e397e9a0857656b0d41f5be2d5f3"} Jul 09 02:05:20.192287 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:20.192239 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-g7cml" podStartSLOduration=2.024970954 podStartE2EDuration="7.192225701s" totalImagesPullingTime="5.167254747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:05:20.191653419 +0000 UTC m=+340.889707232" watchObservedRunningTime="2026-07-09 02:05:20.192225701 +0000 UTC m=+340.890279513" Jul 09 02:05:20.256518 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:20.256454 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:20.311334 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:20.311309 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:21.873310 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:21.873267 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:22.185367 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:22.185335 2582 generic.go:356] "Generic (PLEG): container finished" podID="7e9a20e6-e100-44fe-bc33-77f399a0c262" containerID="d5d5140eeecdb3cff4b3f5d86bce83b79b0074e88feacdd27d7514434e5aab77" exitCode=0 Jul 09 02:05:22.185460 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:22.185402 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh" event={"ID":"7e9a20e6-e100-44fe-bc33-77f399a0c262","Type":"ContainerDied","Data":"d5d5140eeecdb3cff4b3f5d86bce83b79b0074e88feacdd27d7514434e5aab77"} Jul 09 02:05:27.727757 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:27.727712 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:27.831550 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:27.831520 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:30.217066 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:30.217033 2582 generic.go:356] "Generic (PLEG): container finished" podID="7e9a20e6-e100-44fe-bc33-77f399a0c262" containerID="98d3e7c79c552868e2ea8f1ffe5ec858b3b6541deba3a7325e15a5dd68537a8e" exitCode=0 Jul 09 02:05:30.217405 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:30.217094 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh" event={"ID":"7e9a20e6-e100-44fe-bc33-77f399a0c262","Type":"ContainerDied","Data":"98d3e7c79c552868e2ea8f1ffe5ec858b3b6541deba3a7325e15a5dd68537a8e"} Jul 09 02:05:31.340416 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:31.340395 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh" Jul 09 02:05:31.486460 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:31.486384 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e9a20e6-e100-44fe-bc33-77f399a0c262-util\") pod \"7e9a20e6-e100-44fe-bc33-77f399a0c262\" (UID: \"7e9a20e6-e100-44fe-bc33-77f399a0c262\") " Jul 09 02:05:31.486604 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:31.486480 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9px4\" (UniqueName: \"kubernetes.io/projected/7e9a20e6-e100-44fe-bc33-77f399a0c262-kube-api-access-m9px4\") pod \"7e9a20e6-e100-44fe-bc33-77f399a0c262\" (UID: \"7e9a20e6-e100-44fe-bc33-77f399a0c262\") " Jul 09 02:05:31.486604 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:31.486508 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e9a20e6-e100-44fe-bc33-77f399a0c262-bundle\") pod \"7e9a20e6-e100-44fe-bc33-77f399a0c262\" (UID: \"7e9a20e6-e100-44fe-bc33-77f399a0c262\") " Jul 09 02:05:31.486953 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:31.486924 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e9a20e6-e100-44fe-bc33-77f399a0c262-bundle" (OuterVolumeSpecName: "bundle") pod "7e9a20e6-e100-44fe-bc33-77f399a0c262" (UID: "7e9a20e6-e100-44fe-bc33-77f399a0c262"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:05:31.488610 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:31.488583 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e9a20e6-e100-44fe-bc33-77f399a0c262-kube-api-access-m9px4" (OuterVolumeSpecName: "kube-api-access-m9px4") pod "7e9a20e6-e100-44fe-bc33-77f399a0c262" (UID: "7e9a20e6-e100-44fe-bc33-77f399a0c262"). InnerVolumeSpecName "kube-api-access-m9px4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:05:31.490730 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:31.490708 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e9a20e6-e100-44fe-bc33-77f399a0c262-util" (OuterVolumeSpecName: "util") pod "7e9a20e6-e100-44fe-bc33-77f399a0c262" (UID: "7e9a20e6-e100-44fe-bc33-77f399a0c262"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:05:31.586935 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:31.586913 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9px4\" (UniqueName: \"kubernetes.io/projected/7e9a20e6-e100-44fe-bc33-77f399a0c262-kube-api-access-m9px4\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:05:31.586935 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:31.586933 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e9a20e6-e100-44fe-bc33-77f399a0c262-bundle\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:05:31.587056 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:31.586942 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e9a20e6-e100-44fe-bc33-77f399a0c262-util\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:05:32.225577 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:32.225554 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh" Jul 09 02:05:32.225727 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:32.225554 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp5bkh" event={"ID":"7e9a20e6-e100-44fe-bc33-77f399a0c262","Type":"ContainerDied","Data":"d2ba574429864606585f04a8fc42a199a8d183b781265e3e2956bed7d1987fab"} Jul 09 02:05:32.225727 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:32.225680 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2ba574429864606585f04a8fc42a199a8d183b781265e3e2956bed7d1987fab" Jul 09 02:05:33.747866 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:33.747833 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:33.849887 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:33.849850 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:34.898956 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:34.898925 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-km4fn"] Jul 09 02:05:34.899306 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:34.899187 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e9a20e6-e100-44fe-bc33-77f399a0c262" containerName="pull" Jul 09 02:05:34.899306 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:34.899197 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e9a20e6-e100-44fe-bc33-77f399a0c262" containerName="pull" Jul 09 02:05:34.899306 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:34.899205 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e9a20e6-e100-44fe-bc33-77f399a0c262" containerName="util" Jul 09 02:05:34.899306 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:34.899210 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e9a20e6-e100-44fe-bc33-77f399a0c262" containerName="util" Jul 09 02:05:34.899306 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:34.899221 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e9a20e6-e100-44fe-bc33-77f399a0c262" containerName="extract" Jul 09 02:05:34.899306 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:34.899226 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e9a20e6-e100-44fe-bc33-77f399a0c262" containerName="extract" Jul 09 02:05:34.899306 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:34.899268 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e9a20e6-e100-44fe-bc33-77f399a0c262" containerName="extract" Jul 09 02:05:34.946660 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:34.946633 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-km4fn"] Jul 09 02:05:34.946788 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:34.946714 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-km4fn" Jul 09 02:05:34.950254 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:34.950230 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-f2qp8\"" Jul 09 02:05:34.950374 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:34.950233 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 09 02:05:34.951804 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:34.951763 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 09 02:05:35.013077 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:35.013051 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znm8x\" (UniqueName: \"kubernetes.io/projected/8a6b58b4-f622-4762-b72b-21194f7cebc1-kube-api-access-znm8x\") pod \"openshift-lws-operator-fd8ccff4c-km4fn\" (UID: \"8a6b58b4-f622-4762-b72b-21194f7cebc1\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-km4fn" Jul 09 02:05:35.013200 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:35.013095 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a6b58b4-f622-4762-b72b-21194f7cebc1-tmp\") pod \"openshift-lws-operator-fd8ccff4c-km4fn\" (UID: \"8a6b58b4-f622-4762-b72b-21194f7cebc1\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-km4fn" Jul 09 02:05:35.113934 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:35.113910 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znm8x\" (UniqueName: \"kubernetes.io/projected/8a6b58b4-f622-4762-b72b-21194f7cebc1-kube-api-access-znm8x\") pod \"openshift-lws-operator-fd8ccff4c-km4fn\" (UID: \"8a6b58b4-f622-4762-b72b-21194f7cebc1\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-km4fn" Jul 09 02:05:35.114038 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:35.113956 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a6b58b4-f622-4762-b72b-21194f7cebc1-tmp\") pod \"openshift-lws-operator-fd8ccff4c-km4fn\" (UID: \"8a6b58b4-f622-4762-b72b-21194f7cebc1\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-km4fn" Jul 09 02:05:35.114249 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:35.114234 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a6b58b4-f622-4762-b72b-21194f7cebc1-tmp\") pod \"openshift-lws-operator-fd8ccff4c-km4fn\" (UID: \"8a6b58b4-f622-4762-b72b-21194f7cebc1\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-km4fn" Jul 09 02:05:35.122510 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:35.122493 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znm8x\" (UniqueName: \"kubernetes.io/projected/8a6b58b4-f622-4762-b72b-21194f7cebc1-kube-api-access-znm8x\") pod \"openshift-lws-operator-fd8ccff4c-km4fn\" (UID: \"8a6b58b4-f622-4762-b72b-21194f7cebc1\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-km4fn" Jul 09 02:05:35.255860 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:35.255809 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-km4fn" Jul 09 02:05:35.403635 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:35.403576 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-km4fn"] Jul 09 02:05:35.405800 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:05:35.405774 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a6b58b4_f622_4762_b72b_21194f7cebc1.slice/crio-f9cfa3d0e3538593425322c722bf8d72454e946a5ffba14f3e6e8fe7f2015c79 WatchSource:0}: Error finding container f9cfa3d0e3538593425322c722bf8d72454e946a5ffba14f3e6e8fe7f2015c79: Status 404 returned error can't find the container with id f9cfa3d0e3538593425322c722bf8d72454e946a5ffba14f3e6e8fe7f2015c79 Jul 09 02:05:36.240054 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:36.240021 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-km4fn" event={"ID":"8a6b58b4-f622-4762-b72b-21194f7cebc1","Type":"ContainerStarted","Data":"f9cfa3d0e3538593425322c722bf8d72454e946a5ffba14f3e6e8fe7f2015c79"} Jul 09 02:05:39.251209 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:39.251178 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-km4fn" event={"ID":"8a6b58b4-f622-4762-b72b-21194f7cebc1","Type":"ContainerStarted","Data":"60cdbc5acd14d1146b3440de43f61470dc604dd82b1531ec2de11e9baf3907bb"} Jul 09 02:05:39.272095 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:39.272041 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-km4fn" podStartSLOduration=2.301535869 podStartE2EDuration="5.272021083s" totalImagesPullingTime="2.970485214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:05:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:05:39.271250548 +0000 UTC m=+359.969304362" watchObservedRunningTime="2026-07-09 02:05:39.272021083 +0000 UTC m=+359.970074898" Jul 09 02:05:40.002374 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:40.002337 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:40.103431 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:40.103398 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:42.039741 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:42.039710 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:52.290081 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:52.290043 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:52.389072 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:52.389040 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:53.645825 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:53.645790 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:53.749297 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:53.749267 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:54.274920 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.274885 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:54.693451 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.693414 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:54.792039 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.792012 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:54.820118 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.820091 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5"] Jul 09 02:05:54.822351 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.822336 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:05:54.825540 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.825521 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 02:05:54.825665 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.825520 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 02:05:54.825665 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.825600 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k9pj9\"" Jul 09 02:05:54.826031 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.826017 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 02:05:54.826031 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.826026 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 02:05:54.841771 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.841748 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5"] Jul 09 02:05:54.853204 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.853185 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c5cb7ea-589c-41ab-a5ca-54ada4284c7d-tmp\") pod \"opendatahub-operator-controller-manager-8687897cbb-gbnt5\" (UID: \"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d\") " pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:05:54.853294 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.853212 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2c5cb7ea-589c-41ab-a5ca-54ada4284c7d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8687897cbb-gbnt5\" (UID: \"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d\") " pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:05:54.853294 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.853243 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c5cb7ea-589c-41ab-a5ca-54ada4284c7d-webhook-cert\") pod \"opendatahub-operator-controller-manager-8687897cbb-gbnt5\" (UID: \"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d\") " pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:05:54.853373 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.853317 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v97d2\" (UniqueName: \"kubernetes.io/projected/2c5cb7ea-589c-41ab-a5ca-54ada4284c7d-kube-api-access-v97d2\") pod \"opendatahub-operator-controller-manager-8687897cbb-gbnt5\" (UID: \"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d\") " pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:05:54.853373 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.853367 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2c5cb7ea-589c-41ab-a5ca-54ada4284c7d-manifests\") pod \"opendatahub-operator-controller-manager-8687897cbb-gbnt5\" (UID: \"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d\") " pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:05:54.954074 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.954011 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c5cb7ea-589c-41ab-a5ca-54ada4284c7d-tmp\") pod \"opendatahub-operator-controller-manager-8687897cbb-gbnt5\" (UID: \"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d\") " pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:05:54.954074 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.954046 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2c5cb7ea-589c-41ab-a5ca-54ada4284c7d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8687897cbb-gbnt5\" (UID: \"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d\") " pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:05:54.954214 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.954178 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c5cb7ea-589c-41ab-a5ca-54ada4284c7d-webhook-cert\") pod \"opendatahub-operator-controller-manager-8687897cbb-gbnt5\" (UID: \"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d\") " pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:05:54.954259 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.954226 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v97d2\" (UniqueName: \"kubernetes.io/projected/2c5cb7ea-589c-41ab-a5ca-54ada4284c7d-kube-api-access-v97d2\") pod \"opendatahub-operator-controller-manager-8687897cbb-gbnt5\" (UID: \"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d\") " pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:05:54.954307 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.954285 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2c5cb7ea-589c-41ab-a5ca-54ada4284c7d-manifests\") pod \"opendatahub-operator-controller-manager-8687897cbb-gbnt5\" (UID: \"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d\") " pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:05:54.954642 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.954620 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2c5cb7ea-589c-41ab-a5ca-54ada4284c7d-manifests\") pod \"opendatahub-operator-controller-manager-8687897cbb-gbnt5\" (UID: \"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d\") " pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:05:54.956325 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.956307 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c5cb7ea-589c-41ab-a5ca-54ada4284c7d-tmp\") pod \"opendatahub-operator-controller-manager-8687897cbb-gbnt5\" (UID: \"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d\") " pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:05:54.956512 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.956492 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2c5cb7ea-589c-41ab-a5ca-54ada4284c7d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8687897cbb-gbnt5\" (UID: \"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d\") " pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:05:54.956590 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.956576 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c5cb7ea-589c-41ab-a5ca-54ada4284c7d-webhook-cert\") pod \"opendatahub-operator-controller-manager-8687897cbb-gbnt5\" (UID: \"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d\") " pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:05:54.963885 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.963852 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v97d2\" (UniqueName: \"kubernetes.io/projected/2c5cb7ea-589c-41ab-a5ca-54ada4284c7d-kube-api-access-v97d2\") pod \"opendatahub-operator-controller-manager-8687897cbb-gbnt5\" (UID: \"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d\") " pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:05:54.966063 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:54.966045 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:55.069791 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:55.069760 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:55.131925 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:55.131907 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:05:55.167154 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:55.167108 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:55.260298 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:55.260277 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5"] Jul 09 02:05:55.262737 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:05:55.262712 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c5cb7ea_589c_41ab_a5ca_54ada4284c7d.slice/crio-f6f308c113b3b6d3e9f182f1f93a72673cca65efad5e3d0c33b4642816d3aea1 WatchSource:0}: Error finding container f6f308c113b3b6d3e9f182f1f93a72673cca65efad5e3d0c33b4642816d3aea1: Status 404 returned error can't find the container with id f6f308c113b3b6d3e9f182f1f93a72673cca65efad5e3d0c33b4642816d3aea1 Jul 09 02:05:55.265538 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:55.265516 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:55.309387 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:55.309359 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" event={"ID":"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d","Type":"ContainerStarted","Data":"f6f308c113b3b6d3e9f182f1f93a72673cca65efad5e3d0c33b4642816d3aea1"} Jul 09 02:05:55.328829 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:55.328797 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:55.434878 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:55.434846 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:55.530247 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:55.530188 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:55.631595 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:55.631561 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:05:58.326758 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:58.326722 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" event={"ID":"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d","Type":"ContainerStarted","Data":"7fb77739d9b6789791861ef7d63df93fd8ba9f019ba875cb7e7e75549e507c81"} Jul 09 02:05:59.334442 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:59.334417 2582 generic.go:356] "Generic (PLEG): container finished" podID="2c5cb7ea-589c-41ab-a5ca-54ada4284c7d" containerID="7fb77739d9b6789791861ef7d63df93fd8ba9f019ba875cb7e7e75549e507c81" exitCode=0 Jul 09 02:05:59.334749 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:05:59.334479 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" event={"ID":"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d","Type":"ContainerDied","Data":"7fb77739d9b6789791861ef7d63df93fd8ba9f019ba875cb7e7e75549e507c81"} Jul 09 02:06:00.342563 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:00.342528 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" event={"ID":"2c5cb7ea-589c-41ab-a5ca-54ada4284c7d","Type":"ContainerStarted","Data":"2b7849ed4719fe004a3adb71dbb89c5c1464398ecf5f8fd3c8ed43f04bbe9dd5"} Jul 09 02:06:00.342941 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:00.342652 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:06:00.368400 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:00.368354 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" podStartSLOduration=2.108626099 podStartE2EDuration="6.368340322s" totalImagesPullingTime="3.18130918s" totalInitContainerRuntime="1.078405043s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:05:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:06:00.366563059 +0000 UTC m=+381.064616873" watchObservedRunningTime="2026-07-09 02:06:00.368340322 +0000 UTC m=+381.066394132" Jul 09 02:06:11.348901 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:11.348840 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8687897cbb-gbnt5" Jul 09 02:06:14.065862 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:14.065826 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:14.180454 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:14.180406 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:19.708509 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:19.708476 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:19.838785 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:19.838755 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-847fc5468-8lr4q"] Jul 09 02:06:19.844026 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:19.844006 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-847fc5468-8lr4q" Jul 09 02:06:19.847077 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:19.847057 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-l7jkx\"" Jul 09 02:06:19.847189 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:19.847065 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 02:06:19.848828 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:19.848807 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 09 02:06:19.848828 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:19.848827 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 02:06:19.849018 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:19.848807 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 09 02:06:19.852280 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:19.852257 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-847fc5468-8lr4q"] Jul 09 02:06:19.911655 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:19.911619 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:19.950782 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:19.950756 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxk4v\" (UniqueName: \"kubernetes.io/projected/e3f31f39-5c9a-4fda-86fa-331f25d1f555-kube-api-access-wxk4v\") pod \"kube-auth-proxy-847fc5468-8lr4q\" (UID: \"e3f31f39-5c9a-4fda-86fa-331f25d1f555\") " pod="openshift-ingress/kube-auth-proxy-847fc5468-8lr4q" Jul 09 02:06:19.950943 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:19.950790 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3f31f39-5c9a-4fda-86fa-331f25d1f555-tls-certs\") pod \"kube-auth-proxy-847fc5468-8lr4q\" (UID: \"e3f31f39-5c9a-4fda-86fa-331f25d1f555\") " pod="openshift-ingress/kube-auth-proxy-847fc5468-8lr4q" Jul 09 02:06:19.950943 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:19.950807 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3f31f39-5c9a-4fda-86fa-331f25d1f555-tmp\") pod \"kube-auth-proxy-847fc5468-8lr4q\" (UID: \"e3f31f39-5c9a-4fda-86fa-331f25d1f555\") " pod="openshift-ingress/kube-auth-proxy-847fc5468-8lr4q" Jul 09 02:06:20.051473 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.051441 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxk4v\" (UniqueName: \"kubernetes.io/projected/e3f31f39-5c9a-4fda-86fa-331f25d1f555-kube-api-access-wxk4v\") pod \"kube-auth-proxy-847fc5468-8lr4q\" (UID: \"e3f31f39-5c9a-4fda-86fa-331f25d1f555\") " pod="openshift-ingress/kube-auth-proxy-847fc5468-8lr4q" Jul 09 02:06:20.051632 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.051489 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3f31f39-5c9a-4fda-86fa-331f25d1f555-tls-certs\") pod \"kube-auth-proxy-847fc5468-8lr4q\" (UID: \"e3f31f39-5c9a-4fda-86fa-331f25d1f555\") " pod="openshift-ingress/kube-auth-proxy-847fc5468-8lr4q" Jul 09 02:06:20.051632 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.051506 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3f31f39-5c9a-4fda-86fa-331f25d1f555-tmp\") pod \"kube-auth-proxy-847fc5468-8lr4q\" (UID: \"e3f31f39-5c9a-4fda-86fa-331f25d1f555\") " pod="openshift-ingress/kube-auth-proxy-847fc5468-8lr4q" Jul 09 02:06:20.052759 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.052740 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2d6sg"] Jul 09 02:06:20.053739 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.053714 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3f31f39-5c9a-4fda-86fa-331f25d1f555-tmp\") pod \"kube-auth-proxy-847fc5468-8lr4q\" (UID: \"e3f31f39-5c9a-4fda-86fa-331f25d1f555\") " pod="openshift-ingress/kube-auth-proxy-847fc5468-8lr4q" Jul 09 02:06:20.053903 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.053884 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3f31f39-5c9a-4fda-86fa-331f25d1f555-tls-certs\") pod \"kube-auth-proxy-847fc5468-8lr4q\" (UID: \"e3f31f39-5c9a-4fda-86fa-331f25d1f555\") " pod="openshift-ingress/kube-auth-proxy-847fc5468-8lr4q" Jul 09 02:06:20.054997 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.054981 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2d6sg" Jul 09 02:06:20.057958 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.057942 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 09 02:06:20.058117 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.058101 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-944k9\"" Jul 09 02:06:20.064519 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.064500 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2d6sg"] Jul 09 02:06:20.070041 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.070020 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxk4v\" (UniqueName: \"kubernetes.io/projected/e3f31f39-5c9a-4fda-86fa-331f25d1f555-kube-api-access-wxk4v\") pod \"kube-auth-proxy-847fc5468-8lr4q\" (UID: \"e3f31f39-5c9a-4fda-86fa-331f25d1f555\") " pod="openshift-ingress/kube-auth-proxy-847fc5468-8lr4q" Jul 09 02:06:20.102509 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.102481 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:20.152449 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.152422 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dlfr\" (UniqueName: \"kubernetes.io/projected/06e1f2d4-cf8d-4d3b-8123-4741b828125f-kube-api-access-6dlfr\") pod \"odh-model-controller-7998c67dc4-2d6sg\" (UID: \"06e1f2d4-cf8d-4d3b-8123-4741b828125f\") " pod="opendatahub/odh-model-controller-7998c67dc4-2d6sg" Jul 09 02:06:20.152599 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.152482 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06e1f2d4-cf8d-4d3b-8123-4741b828125f-cert\") pod \"odh-model-controller-7998c67dc4-2d6sg\" (UID: \"06e1f2d4-cf8d-4d3b-8123-4741b828125f\") " pod="opendatahub/odh-model-controller-7998c67dc4-2d6sg" Jul 09 02:06:20.155440 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.155422 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-847fc5468-8lr4q" Jul 09 02:06:20.205290 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.205261 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:20.253482 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.253398 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06e1f2d4-cf8d-4d3b-8123-4741b828125f-cert\") pod \"odh-model-controller-7998c67dc4-2d6sg\" (UID: \"06e1f2d4-cf8d-4d3b-8123-4741b828125f\") " pod="opendatahub/odh-model-controller-7998c67dc4-2d6sg" Jul 09 02:06:20.253625 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.253502 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dlfr\" (UniqueName: \"kubernetes.io/projected/06e1f2d4-cf8d-4d3b-8123-4741b828125f-kube-api-access-6dlfr\") pod \"odh-model-controller-7998c67dc4-2d6sg\" (UID: \"06e1f2d4-cf8d-4d3b-8123-4741b828125f\") " pod="opendatahub/odh-model-controller-7998c67dc4-2d6sg" Jul 09 02:06:20.256406 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.256384 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06e1f2d4-cf8d-4d3b-8123-4741b828125f-cert\") pod \"odh-model-controller-7998c67dc4-2d6sg\" (UID: \"06e1f2d4-cf8d-4d3b-8123-4741b828125f\") " pod="opendatahub/odh-model-controller-7998c67dc4-2d6sg" Jul 09 02:06:20.262550 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.262527 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dlfr\" (UniqueName: \"kubernetes.io/projected/06e1f2d4-cf8d-4d3b-8123-4741b828125f-kube-api-access-6dlfr\") pod \"odh-model-controller-7998c67dc4-2d6sg\" (UID: \"06e1f2d4-cf8d-4d3b-8123-4741b828125f\") " pod="opendatahub/odh-model-controller-7998c67dc4-2d6sg" Jul 09 02:06:20.284375 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.284354 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-847fc5468-8lr4q"] Jul 09 02:06:20.286654 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:06:20.286621 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3f31f39_5c9a_4fda_86fa_331f25d1f555.slice/crio-41d08a12c0cd898fa73c6948355f8fa46b036bd14c024953494bc2dff53442c6 WatchSource:0}: Error finding container 41d08a12c0cd898fa73c6948355f8fa46b036bd14c024953494bc2dff53442c6: Status 404 returned error can't find the container with id 41d08a12c0cd898fa73c6948355f8fa46b036bd14c024953494bc2dff53442c6 Jul 09 02:06:20.290636 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.290612 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:20.382838 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.382806 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2d6sg" Jul 09 02:06:20.422188 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.422157 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-847fc5468-8lr4q" event={"ID":"e3f31f39-5c9a-4fda-86fa-331f25d1f555","Type":"ContainerStarted","Data":"41d08a12c0cd898fa73c6948355f8fa46b036bd14c024953494bc2dff53442c6"} Jul 09 02:06:20.550760 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.550737 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2d6sg"] Jul 09 02:06:20.553590 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:06:20.553555 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06e1f2d4_cf8d_4d3b_8123_4741b828125f.slice/crio-00d544912af6ba4d262da4fcd90325f10abaed6a5f9ab3198387a91c91d9b626 WatchSource:0}: Error finding container 00d544912af6ba4d262da4fcd90325f10abaed6a5f9ab3198387a91c91d9b626: Status 404 returned error can't find the container with id 00d544912af6ba4d262da4fcd90325f10abaed6a5f9ab3198387a91c91d9b626 Jul 09 02:06:20.642219 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.642186 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:20.684463 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.684436 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8cstw"] Jul 09 02:06:20.687344 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.687323 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-8cstw" Jul 09 02:06:20.690131 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.690113 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 09 02:06:20.698572 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.698548 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8cstw"] Jul 09 02:06:20.759101 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.759067 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlhct\" (UniqueName: \"kubernetes.io/projected/fc976687-df22-4397-a0dd-797c604bc7a6-kube-api-access-nlhct\") pod \"model-serving-api-54d89bd5fc-8cstw\" (UID: \"fc976687-df22-4397-a0dd-797c604bc7a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-8cstw" Jul 09 02:06:20.759534 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.759140 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc976687-df22-4397-a0dd-797c604bc7a6-tls-certs\") pod \"model-serving-api-54d89bd5fc-8cstw\" (UID: \"fc976687-df22-4397-a0dd-797c604bc7a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-8cstw" Jul 09 02:06:20.860346 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.860252 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc976687-df22-4397-a0dd-797c604bc7a6-tls-certs\") pod \"model-serving-api-54d89bd5fc-8cstw\" (UID: \"fc976687-df22-4397-a0dd-797c604bc7a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-8cstw" Jul 09 02:06:20.860346 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.860328 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlhct\" (UniqueName: \"kubernetes.io/projected/fc976687-df22-4397-a0dd-797c604bc7a6-kube-api-access-nlhct\") pod \"model-serving-api-54d89bd5fc-8cstw\" (UID: \"fc976687-df22-4397-a0dd-797c604bc7a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-8cstw" Jul 09 02:06:20.862670 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.862640 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc976687-df22-4397-a0dd-797c604bc7a6-tls-certs\") pod \"model-serving-api-54d89bd5fc-8cstw\" (UID: \"fc976687-df22-4397-a0dd-797c604bc7a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-8cstw" Jul 09 02:06:20.868500 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.868477 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlhct\" (UniqueName: \"kubernetes.io/projected/fc976687-df22-4397-a0dd-797c604bc7a6-kube-api-access-nlhct\") pod \"model-serving-api-54d89bd5fc-8cstw\" (UID: \"fc976687-df22-4397-a0dd-797c604bc7a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-8cstw" Jul 09 02:06:20.877300 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.877282 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:20.998751 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:20.998688 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-8cstw" Jul 09 02:06:21.028659 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.028617 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:21.036200 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.036174 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs"] Jul 09 02:06:21.039195 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.039171 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.042354 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.042214 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 09 02:06:21.042524 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.042503 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 09 02:06:21.042643 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.042600 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 09 02:06:21.042720 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.042705 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-8dn2t\"" Jul 09 02:06:21.042795 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.042782 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 02:06:21.051058 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.051034 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs"] Jul 09 02:06:21.110538 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.110416 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:21.136925 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.136897 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8cstw"] Jul 09 02:06:21.139332 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:06:21.139296 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc976687_df22_4397_a0dd_797c604bc7a6.slice/crio-85a88201c72b551a24db8ea0dde458ed958cab8173e8b65ba00f83d8047a94b1 WatchSource:0}: Error finding container 85a88201c72b551a24db8ea0dde458ed958cab8173e8b65ba00f83d8047a94b1: Status 404 returned error can't find the container with id 85a88201c72b551a24db8ea0dde458ed958cab8173e8b65ba00f83d8047a94b1 Jul 09 02:06:21.164429 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.164407 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7d374966-94fd-49e2-8efc-3a76b35e5bf9-cacerts\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.164539 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.164439 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.164539 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.164464 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.164623 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.164578 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4wx8\" (UniqueName: \"kubernetes.io/projected/7d374966-94fd-49e2-8efc-3a76b35e5bf9-kube-api-access-k4wx8\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.164673 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.164618 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.164673 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.164645 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d374966-94fd-49e2-8efc-3a76b35e5bf9-local-certs\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.164673 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.164667 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-token\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.219019 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.218989 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:21.265301 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.265272 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4wx8\" (UniqueName: \"kubernetes.io/projected/7d374966-94fd-49e2-8efc-3a76b35e5bf9-kube-api-access-k4wx8\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.265426 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.265317 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.265426 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.265345 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d374966-94fd-49e2-8efc-3a76b35e5bf9-local-certs\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.265426 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.265373 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-token\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.265596 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.265440 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7d374966-94fd-49e2-8efc-3a76b35e5bf9-cacerts\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.265596 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.265466 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.265596 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.265497 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.267145 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.266570 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.268862 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.268827 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d374966-94fd-49e2-8efc-3a76b35e5bf9-local-certs\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.269251 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.269015 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7d374966-94fd-49e2-8efc-3a76b35e5bf9-cacerts\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.269396 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.269353 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.269508 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.269479 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.275100 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.275075 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4wx8\" (UniqueName: \"kubernetes.io/projected/7d374966-94fd-49e2-8efc-3a76b35e5bf9-kube-api-access-k4wx8\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.275516 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.275497 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-token\") pod \"istiod-openshift-gateway-7574459656-2mkbs\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.351336 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.351309 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:21.437102 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.437007 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2d6sg" event={"ID":"06e1f2d4-cf8d-4d3b-8123-4741b828125f","Type":"ContainerStarted","Data":"00d544912af6ba4d262da4fcd90325f10abaed6a5f9ab3198387a91c91d9b626"} Jul 09 02:06:21.442264 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.442232 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-8cstw" event={"ID":"fc976687-df22-4397-a0dd-797c604bc7a6","Type":"ContainerStarted","Data":"85a88201c72b551a24db8ea0dde458ed958cab8173e8b65ba00f83d8047a94b1"} Jul 09 02:06:21.530666 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:21.530637 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs"] Jul 09 02:06:22.255446 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:22.255412 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:22.464721 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:22.464662 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" event={"ID":"7d374966-94fd-49e2-8efc-3a76b35e5bf9","Type":"ContainerStarted","Data":"768670b928ffc52ed888f41e0a59a07143716fdd62e353ff4f9f4a9f18a75cd5"} Jul 09 02:06:25.979805 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:25.979774 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:06:25.980123 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:25.979835 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:06:26.487801 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:26.487677 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-847fc5468-8lr4q" event={"ID":"e3f31f39-5c9a-4fda-86fa-331f25d1f555","Type":"ContainerStarted","Data":"7610f7a6baebcda4c9d4a6262688f208e67c4be1081b29bf2764eb1820674549"} Jul 09 02:06:26.489476 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:26.489447 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2d6sg" event={"ID":"06e1f2d4-cf8d-4d3b-8123-4741b828125f","Type":"ContainerStarted","Data":"317e6585ce69a1eb0a4941720c36efdc95836d69b944551e83169fe8337f2b05"} Jul 09 02:06:26.489633 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:26.489556 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-2d6sg" Jul 09 02:06:26.491180 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:26.491154 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-8cstw" event={"ID":"fc976687-df22-4397-a0dd-797c604bc7a6","Type":"ContainerStarted","Data":"bcaeca05ff2d196c9cfa3a917e88db855920a891339a5943260deb93bfa1c68d"} Jul 09 02:06:26.491347 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:26.491317 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-8cstw" Jul 09 02:06:26.492911 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:26.492888 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" event={"ID":"7d374966-94fd-49e2-8efc-3a76b35e5bf9","Type":"ContainerStarted","Data":"1725001595ca7c879d3fa2da8348ee0393f4d18d0bb7c310a5f2172cb49605fc"} Jul 09 02:06:26.493019 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:26.493005 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:26.505601 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:26.505554 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-847fc5468-8lr4q" podStartSLOduration=1.821059177 podStartE2EDuration="7.505541997s" totalImagesPullingTime="5.68448282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:06:26.505051546 +0000 UTC m=+407.203105360" watchObservedRunningTime="2026-07-09 02:06:26.505541997 +0000 UTC m=+407.203595809" Jul 09 02:06:26.520777 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:26.520725 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-2d6sg" podStartSLOduration=1.086136754 podStartE2EDuration="6.520707828s" totalImagesPullingTime="5.434571074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:06:26.520020089 +0000 UTC m=+407.218073903" watchObservedRunningTime="2026-07-09 02:06:26.520707828 +0000 UTC m=+407.218761642" Jul 09 02:06:26.542321 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:26.542277 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" podStartSLOduration=1.093587564 podStartE2EDuration="5.542264266s" totalImagesPullingTime="4.448676702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:06:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:06:26.542229913 +0000 UTC m=+407.240283728" watchObservedRunningTime="2026-07-09 02:06:26.542264266 +0000 UTC m=+407.240318080" Jul 09 02:06:26.561148 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:26.561106 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-8cstw" podStartSLOduration=1.729792936 podStartE2EDuration="6.561092921s" totalImagesPullingTime="4.831299985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:06:26.55930218 +0000 UTC m=+407.257355994" watchObservedRunningTime="2026-07-09 02:06:26.561092921 +0000 UTC m=+407.259146733" Jul 09 02:06:27.001234 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:27.001184 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:27.064663 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:27.064619 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:27.498458 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:27.498431 2582 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-2mkbs container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 02:06:27.498604 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:27.498477 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" podUID="7d374966-94fd-49e2-8efc-3a76b35e5bf9" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 02:06:27.727368 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:27.727334 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:27.998911 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:27.998863 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:28.101794 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:28.101761 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:30.498249 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:30.498215 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:31.556204 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.556170 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:31.611469 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.611436 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-rkl5n"] Jul 09 02:06:31.613625 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.613605 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rkl5n" Jul 09 02:06:31.618321 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.618299 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 09 02:06:31.618321 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.618315 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-cdvft\"" Jul 09 02:06:31.619974 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.619956 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dnrfl"] Jul 09 02:06:31.622304 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.622289 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dnrfl" Jul 09 02:06:31.625642 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.625620 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-dkbnf\"" Jul 09 02:06:31.625813 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.625796 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 09 02:06:31.626992 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.626973 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-rkl5n"] Jul 09 02:06:31.634095 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.634077 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dnrfl"] Jul 09 02:06:31.647719 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.647696 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:31.728690 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.728667 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:31.753931 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.753911 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr9j5\" (UniqueName: \"kubernetes.io/projected/433ba04e-75c0-4ce1-8793-aec0e8ff93d6-kube-api-access-xr9j5\") pod \"llmisvc-controller-manager-777c6bd7d5-dnrfl\" (UID: \"433ba04e-75c0-4ce1-8793-aec0e8ff93d6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dnrfl" Jul 09 02:06:31.754009 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.753942 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rhfh\" (UniqueName: \"kubernetes.io/projected/bf4f6c90-4abd-4907-b6cb-58b8f67af948-kube-api-access-7rhfh\") pod \"kserve-controller-manager-d6f4b65b8-rkl5n\" (UID: \"bf4f6c90-4abd-4907-b6cb-58b8f67af948\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rkl5n" Jul 09 02:06:31.754009 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.753991 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/433ba04e-75c0-4ce1-8793-aec0e8ff93d6-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dnrfl\" (UID: \"433ba04e-75c0-4ce1-8793-aec0e8ff93d6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dnrfl" Jul 09 02:06:31.754072 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.754015 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf4f6c90-4abd-4907-b6cb-58b8f67af948-cert\") pod \"kserve-controller-manager-d6f4b65b8-rkl5n\" (UID: \"bf4f6c90-4abd-4907-b6cb-58b8f67af948\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rkl5n" Jul 09 02:06:31.854688 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.854625 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/433ba04e-75c0-4ce1-8793-aec0e8ff93d6-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dnrfl\" (UID: \"433ba04e-75c0-4ce1-8793-aec0e8ff93d6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dnrfl" Jul 09 02:06:31.854688 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.854658 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf4f6c90-4abd-4907-b6cb-58b8f67af948-cert\") pod \"kserve-controller-manager-d6f4b65b8-rkl5n\" (UID: \"bf4f6c90-4abd-4907-b6cb-58b8f67af948\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rkl5n" Jul 09 02:06:31.854854 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.854693 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr9j5\" (UniqueName: \"kubernetes.io/projected/433ba04e-75c0-4ce1-8793-aec0e8ff93d6-kube-api-access-xr9j5\") pod \"llmisvc-controller-manager-777c6bd7d5-dnrfl\" (UID: \"433ba04e-75c0-4ce1-8793-aec0e8ff93d6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dnrfl" Jul 09 02:06:31.854854 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.854810 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rhfh\" (UniqueName: \"kubernetes.io/projected/bf4f6c90-4abd-4907-b6cb-58b8f67af948-kube-api-access-7rhfh\") pod \"kserve-controller-manager-d6f4b65b8-rkl5n\" (UID: \"bf4f6c90-4abd-4907-b6cb-58b8f67af948\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rkl5n" Jul 09 02:06:31.855146 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:06:31.854770 2582 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 09 02:06:31.855222 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:06:31.855210 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/433ba04e-75c0-4ce1-8793-aec0e8ff93d6-cert podName:433ba04e-75c0-4ce1-8793-aec0e8ff93d6 nodeName:}" failed. No retries permitted until 2026-07-09 02:06:32.355183648 +0000 UTC m=+413.053237499 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/433ba04e-75c0-4ce1-8793-aec0e8ff93d6-cert") pod "llmisvc-controller-manager-777c6bd7d5-dnrfl" (UID: "433ba04e-75c0-4ce1-8793-aec0e8ff93d6") : secret "llmisvc-webhook-server-cert" not found Jul 09 02:06:31.861295 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.861275 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf4f6c90-4abd-4907-b6cb-58b8f67af948-cert\") pod \"kserve-controller-manager-d6f4b65b8-rkl5n\" (UID: \"bf4f6c90-4abd-4907-b6cb-58b8f67af948\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rkl5n" Jul 09 02:06:31.867640 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.867612 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rhfh\" (UniqueName: \"kubernetes.io/projected/bf4f6c90-4abd-4907-b6cb-58b8f67af948-kube-api-access-7rhfh\") pod \"kserve-controller-manager-d6f4b65b8-rkl5n\" (UID: \"bf4f6c90-4abd-4907-b6cb-58b8f67af948\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rkl5n" Jul 09 02:06:31.867948 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.867929 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr9j5\" (UniqueName: \"kubernetes.io/projected/433ba04e-75c0-4ce1-8793-aec0e8ff93d6-kube-api-access-xr9j5\") pod \"llmisvc-controller-manager-777c6bd7d5-dnrfl\" (UID: \"433ba04e-75c0-4ce1-8793-aec0e8ff93d6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dnrfl" Jul 09 02:06:31.924441 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:31.924419 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rkl5n" Jul 09 02:06:32.053003 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:32.052980 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-rkl5n"] Jul 09 02:06:32.055000 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:06:32.054975 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf4f6c90_4abd_4907_b6cb_58b8f67af948.slice/crio-5427c636deecc9216fd34d917b2071cbf89f91c728f35fb0a83a59f6ac1900d8 WatchSource:0}: Error finding container 5427c636deecc9216fd34d917b2071cbf89f91c728f35fb0a83a59f6ac1900d8: Status 404 returned error can't find the container with id 5427c636deecc9216fd34d917b2071cbf89f91c728f35fb0a83a59f6ac1900d8 Jul 09 02:06:32.058305 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:32.058278 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:32.358758 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:32.358733 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/433ba04e-75c0-4ce1-8793-aec0e8ff93d6-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dnrfl\" (UID: \"433ba04e-75c0-4ce1-8793-aec0e8ff93d6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dnrfl" Jul 09 02:06:32.361061 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:32.361039 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/433ba04e-75c0-4ce1-8793-aec0e8ff93d6-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dnrfl\" (UID: \"433ba04e-75c0-4ce1-8793-aec0e8ff93d6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dnrfl" Jul 09 02:06:32.517428 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:32.517400 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rkl5n" event={"ID":"bf4f6c90-4abd-4907-b6cb-58b8f67af948","Type":"ContainerStarted","Data":"5427c636deecc9216fd34d917b2071cbf89f91c728f35fb0a83a59f6ac1900d8"} Jul 09 02:06:32.533631 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:32.533609 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dnrfl" Jul 09 02:06:32.662996 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:32.662963 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dnrfl"] Jul 09 02:06:32.664837 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:06:32.664812 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod433ba04e_75c0_4ce1_8793_aec0e8ff93d6.slice/crio-7cc6413f80fcaf54a10877dc72dbaa5e0d6b46c5ca623785ce9adb351e535846 WatchSource:0}: Error finding container 7cc6413f80fcaf54a10877dc72dbaa5e0d6b46c5ca623785ce9adb351e535846: Status 404 returned error can't find the container with id 7cc6413f80fcaf54a10877dc72dbaa5e0d6b46c5ca623785ce9adb351e535846 Jul 09 02:06:32.842830 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:32.842797 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:32.945986 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:32.945951 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:33.026355 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:33.026319 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:06:33.525432 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:33.525401 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dnrfl" event={"ID":"433ba04e-75c0-4ce1-8793-aec0e8ff93d6","Type":"ContainerStarted","Data":"7cc6413f80fcaf54a10877dc72dbaa5e0d6b46c5ca623785ce9adb351e535846"} Jul 09 02:06:34.437492 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:34.437463 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs"] Jul 09 02:06:34.437906 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:34.437701 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" podUID="7d374966-94fd-49e2-8efc-3a76b35e5bf9" containerName="discovery" containerID="cri-o://1725001595ca7c879d3fa2da8348ee0393f4d18d0bb7c310a5f2172cb49605fc" gracePeriod=30 Jul 09 02:06:35.537358 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.537324 2582 generic.go:356] "Generic (PLEG): container finished" podID="7d374966-94fd-49e2-8efc-3a76b35e5bf9" containerID="1725001595ca7c879d3fa2da8348ee0393f4d18d0bb7c310a5f2172cb49605fc" exitCode=0 Jul 09 02:06:35.537358 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.537357 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" event={"ID":"7d374966-94fd-49e2-8efc-3a76b35e5bf9","Type":"ContainerDied","Data":"1725001595ca7c879d3fa2da8348ee0393f4d18d0bb7c310a5f2172cb49605fc"} Jul 09 02:06:35.605239 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.605215 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:35.790114 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.790062 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-token\") pod \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " Jul 09 02:06:35.790114 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.790111 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-kubeconfig\") pod \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " Jul 09 02:06:35.790238 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.790151 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7d374966-94fd-49e2-8efc-3a76b35e5bf9-cacerts\") pod \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " Jul 09 02:06:35.790238 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.790193 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d374966-94fd-49e2-8efc-3a76b35e5bf9-local-certs\") pod \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " Jul 09 02:06:35.790238 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.790231 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-csr-dns-cert\") pod \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " Jul 09 02:06:35.790366 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.790264 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4wx8\" (UniqueName: \"kubernetes.io/projected/7d374966-94fd-49e2-8efc-3a76b35e5bf9-kube-api-access-k4wx8\") pod \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " Jul 09 02:06:35.790366 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.790285 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-csr-ca-configmap\") pod \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\" (UID: \"7d374966-94fd-49e2-8efc-3a76b35e5bf9\") " Jul 09 02:06:35.790753 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.790726 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "7d374966-94fd-49e2-8efc-3a76b35e5bf9" (UID: "7d374966-94fd-49e2-8efc-3a76b35e5bf9"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 02:06:35.792777 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.792752 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d374966-94fd-49e2-8efc-3a76b35e5bf9-cacerts" (OuterVolumeSpecName: "cacerts") pod "7d374966-94fd-49e2-8efc-3a76b35e5bf9" (UID: "7d374966-94fd-49e2-8efc-3a76b35e5bf9"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 02:06:35.792859 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.792775 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-token" (OuterVolumeSpecName: "istio-token") pod "7d374966-94fd-49e2-8efc-3a76b35e5bf9" (UID: "7d374966-94fd-49e2-8efc-3a76b35e5bf9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:06:35.792859 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.792819 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d374966-94fd-49e2-8efc-3a76b35e5bf9-kube-api-access-k4wx8" (OuterVolumeSpecName: "kube-api-access-k4wx8") pod "7d374966-94fd-49e2-8efc-3a76b35e5bf9" (UID: "7d374966-94fd-49e2-8efc-3a76b35e5bf9"). InnerVolumeSpecName "kube-api-access-k4wx8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:06:35.792859 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.792828 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d374966-94fd-49e2-8efc-3a76b35e5bf9-local-certs" (OuterVolumeSpecName: "local-certs") pod "7d374966-94fd-49e2-8efc-3a76b35e5bf9" (UID: "7d374966-94fd-49e2-8efc-3a76b35e5bf9"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:06:35.792859 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.792816 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "7d374966-94fd-49e2-8efc-3a76b35e5bf9" (UID: "7d374966-94fd-49e2-8efc-3a76b35e5bf9"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 02:06:35.792859 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.792839 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "7d374966-94fd-49e2-8efc-3a76b35e5bf9" (UID: "7d374966-94fd-49e2-8efc-3a76b35e5bf9"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 02:06:35.890835 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.890798 2582 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d374966-94fd-49e2-8efc-3a76b35e5bf9-local-certs\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:06:35.890835 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.890830 2582 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-csr-dns-cert\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:06:35.891023 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.890845 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4wx8\" (UniqueName: \"kubernetes.io/projected/7d374966-94fd-49e2-8efc-3a76b35e5bf9-kube-api-access-k4wx8\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:06:35.891023 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.890858 2582 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-csr-ca-configmap\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:06:35.891023 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.890888 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-token\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:06:35.891023 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.890901 2582 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d374966-94fd-49e2-8efc-3a76b35e5bf9-istio-kubeconfig\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:06:35.891023 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:35.890913 2582 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7d374966-94fd-49e2-8efc-3a76b35e5bf9-cacerts\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:06:35.901386 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:06:35.901360 2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d374966_94fd_49e2_8efc_3a76b35e5bf9.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d374966_94fd_49e2_8efc_3a76b35e5bf9.slice/crio-768670b928ffc52ed888f41e0a59a07143716fdd62e353ff4f9f4a9f18a75cd5\": RecentStats: unable to find data in memory cache]" Jul 09 02:06:36.543521 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:36.543478 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rkl5n" event={"ID":"bf4f6c90-4abd-4907-b6cb-58b8f67af948","Type":"ContainerStarted","Data":"d229db406defd336a6d736ae8785b5826cf439293ad8fb7bcb7d5e869023d943"} Jul 09 02:06:36.543995 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:36.543614 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rkl5n" Jul 09 02:06:36.545073 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:36.545050 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dnrfl" event={"ID":"433ba04e-75c0-4ce1-8793-aec0e8ff93d6","Type":"ContainerStarted","Data":"926e56e0d09b33c1504667d2621c885765bb9b5533f2d8a5ea34e07fd93d64da"} Jul 09 02:06:36.545145 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:36.545102 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dnrfl" Jul 09 02:06:36.546252 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:36.546230 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" event={"ID":"7d374966-94fd-49e2-8efc-3a76b35e5bf9","Type":"ContainerDied","Data":"768670b928ffc52ed888f41e0a59a07143716fdd62e353ff4f9f4a9f18a75cd5"} Jul 09 02:06:36.546339 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:36.546255 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs" Jul 09 02:06:36.546339 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:36.546281 2582 scope.go:117] "RemoveContainer" containerID="1725001595ca7c879d3fa2da8348ee0393f4d18d0bb7c310a5f2172cb49605fc" Jul 09 02:06:36.572057 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:36.572013 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rkl5n" podStartSLOduration=2.008295684 podStartE2EDuration="5.572001903s" totalImagesPullingTime="3.563706219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:06:36.570799633 +0000 UTC m=+417.268853457" watchObservedRunningTime="2026-07-09 02:06:36.572001903 +0000 UTC m=+417.270055715" Jul 09 02:06:36.614294 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:36.614244 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dnrfl" podStartSLOduration=2.656795213 podStartE2EDuration="5.614229744s" totalImagesPullingTime="2.957434531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:06:36.612533018 +0000 UTC m=+417.310586830" watchObservedRunningTime="2026-07-09 02:06:36.614229744 +0000 UTC m=+417.312283596" Jul 09 02:06:36.637169 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:36.637144 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs"] Jul 09 02:06:36.641127 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:36.641105 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2mkbs"] Jul 09 02:06:37.498284 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:37.498261 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-2d6sg" Jul 09 02:06:37.501605 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:37.501584 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-8cstw" Jul 09 02:06:37.865131 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:37.865049 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d374966-94fd-49e2-8efc-3a76b35e5bf9" path="/var/lib/kubelet/pods/7d374966-94fd-49e2-8efc-3a76b35e5bf9/volumes" Jul 09 02:06:54.193477 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:06:54.193440 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:07:07.555264 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:07.555195 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dnrfl" Jul 09 02:07:07.558275 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:07.558255 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rkl5n" Jul 09 02:07:15.655074 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:15.655033 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:07:15.752028 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:15.751995 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:07:15.781300 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:15.781275 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7s762"] Jul 09 02:07:15.781560 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:15.781548 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d374966-94fd-49e2-8efc-3a76b35e5bf9" containerName="discovery" Jul 09 02:07:15.781560 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:15.781561 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d374966-94fd-49e2-8efc-3a76b35e5bf9" containerName="discovery" Jul 09 02:07:15.781655 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:15.781623 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d374966-94fd-49e2-8efc-3a76b35e5bf9" containerName="discovery" Jul 09 02:07:15.786346 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:15.786325 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7s762" Jul 09 02:07:15.789554 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:15.789530 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 02:07:15.791107 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:15.791085 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 02:07:15.791198 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:15.791088 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-7w7dd\"" Jul 09 02:07:15.794711 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:15.794688 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7s762"] Jul 09 02:07:15.833652 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:15.833623 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:07:15.870919 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:15.870896 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmrd5\" (UniqueName: \"kubernetes.io/projected/491384e1-82a7-4067-975b-d50e947d4acf-kube-api-access-bmrd5\") pod \"kuadrant-operator-catalog-7s762\" (UID: \"491384e1-82a7-4067-975b-d50e947d4acf\") " pod="kuadrant-system/kuadrant-operator-catalog-7s762" Jul 09 02:07:15.971861 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:15.971782 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmrd5\" (UniqueName: \"kubernetes.io/projected/491384e1-82a7-4067-975b-d50e947d4acf-kube-api-access-bmrd5\") pod \"kuadrant-operator-catalog-7s762\" (UID: \"491384e1-82a7-4067-975b-d50e947d4acf\") " pod="kuadrant-system/kuadrant-operator-catalog-7s762" Jul 09 02:07:15.982660 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:15.982639 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmrd5\" (UniqueName: \"kubernetes.io/projected/491384e1-82a7-4067-975b-d50e947d4acf-kube-api-access-bmrd5\") pod \"kuadrant-operator-catalog-7s762\" (UID: \"491384e1-82a7-4067-975b-d50e947d4acf\") " pod="kuadrant-system/kuadrant-operator-catalog-7s762" Jul 09 02:07:16.097429 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:16.097395 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7s762" Jul 09 02:07:16.151441 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:16.151393 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7s762"] Jul 09 02:07:16.224282 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:16.224209 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7s762"] Jul 09 02:07:16.227626 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:07:16.227598 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod491384e1_82a7_4067_975b_d50e947d4acf.slice/crio-2d36ccfd81397e7ef0a11b25affdd20de70ff77eb1d69ab8ce7f388b6ca4d5ec WatchSource:0}: Error finding container 2d36ccfd81397e7ef0a11b25affdd20de70ff77eb1d69ab8ce7f388b6ca4d5ec: Status 404 returned error can't find the container with id 2d36ccfd81397e7ef0a11b25affdd20de70ff77eb1d69ab8ce7f388b6ca4d5ec Jul 09 02:07:16.359695 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:16.359666 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-52mq2"] Jul 09 02:07:16.362511 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:16.362492 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-52mq2" Jul 09 02:07:16.371083 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:16.371061 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-52mq2"] Jul 09 02:07:16.476052 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:16.475975 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2bnl\" (UniqueName: \"kubernetes.io/projected/9137fc0a-da4d-4f44-a462-32a6a81e8571-kube-api-access-f2bnl\") pod \"kuadrant-operator-catalog-52mq2\" (UID: \"9137fc0a-da4d-4f44-a462-32a6a81e8571\") " pod="kuadrant-system/kuadrant-operator-catalog-52mq2" Jul 09 02:07:16.577256 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:16.577223 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2bnl\" (UniqueName: \"kubernetes.io/projected/9137fc0a-da4d-4f44-a462-32a6a81e8571-kube-api-access-f2bnl\") pod \"kuadrant-operator-catalog-52mq2\" (UID: \"9137fc0a-da4d-4f44-a462-32a6a81e8571\") " pod="kuadrant-system/kuadrant-operator-catalog-52mq2" Jul 09 02:07:16.585648 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:16.585628 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2bnl\" (UniqueName: \"kubernetes.io/projected/9137fc0a-da4d-4f44-a462-32a6a81e8571-kube-api-access-f2bnl\") pod \"kuadrant-operator-catalog-52mq2\" (UID: \"9137fc0a-da4d-4f44-a462-32a6a81e8571\") " pod="kuadrant-system/kuadrant-operator-catalog-52mq2" Jul 09 02:07:16.672683 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:16.672653 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-52mq2" Jul 09 02:07:16.720906 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:16.720847 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7s762" event={"ID":"491384e1-82a7-4067-975b-d50e947d4acf","Type":"ContainerStarted","Data":"2d36ccfd81397e7ef0a11b25affdd20de70ff77eb1d69ab8ce7f388b6ca4d5ec"} Jul 09 02:07:16.804251 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:16.804227 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-52mq2"] Jul 09 02:07:17.727403 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:17.727365 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-52mq2" event={"ID":"9137fc0a-da4d-4f44-a462-32a6a81e8571","Type":"ContainerStarted","Data":"6a6e9f4e02733b57ce40bd54b13992412f20fd98d65004e87e6933bb083740ef"} Jul 09 02:07:18.733428 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:18.733392 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7s762" event={"ID":"491384e1-82a7-4067-975b-d50e947d4acf","Type":"ContainerStarted","Data":"96200853f0a03e2a619aad1de9b2023198b9ffdcccb2182fe61eaf2357caa099"} Jul 09 02:07:18.733864 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:18.733432 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-7s762" podUID="491384e1-82a7-4067-975b-d50e947d4acf" containerName="registry-server" containerID="cri-o://96200853f0a03e2a619aad1de9b2023198b9ffdcccb2182fe61eaf2357caa099" gracePeriod=2 Jul 09 02:07:18.734901 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:18.734859 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-52mq2" event={"ID":"9137fc0a-da4d-4f44-a462-32a6a81e8571","Type":"ContainerStarted","Data":"4b1a17591ba22edf98d45ec6ffb7c27e09b5166f339477cad7691d81cef919a6"} Jul 09 02:07:18.777291 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:18.777238 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-7s762" podStartSLOduration=1.800998141 podStartE2EDuration="3.777218805s" totalImagesPullingTime="1.976220664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:07:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:07:18.777178721 +0000 UTC m=+459.475232536" watchObservedRunningTime="2026-07-09 02:07:18.777218805 +0000 UTC m=+459.475272619" Jul 09 02:07:18.977881 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:18.977845 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7s762" Jul 09 02:07:18.994706 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:18.994626 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-52mq2" podStartSLOduration=1.6131697539999998 podStartE2EDuration="2.994609576s" totalImagesPullingTime="1.381439822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:07:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:07:18.804990695 +0000 UTC m=+459.503044508" watchObservedRunningTime="2026-07-09 02:07:18.994609576 +0000 UTC m=+459.692663390" Jul 09 02:07:19.100052 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:19.100017 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmrd5\" (UniqueName: \"kubernetes.io/projected/491384e1-82a7-4067-975b-d50e947d4acf-kube-api-access-bmrd5\") pod \"491384e1-82a7-4067-975b-d50e947d4acf\" (UID: \"491384e1-82a7-4067-975b-d50e947d4acf\") " Jul 09 02:07:19.102133 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:19.102107 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/491384e1-82a7-4067-975b-d50e947d4acf-kube-api-access-bmrd5" (OuterVolumeSpecName: "kube-api-access-bmrd5") pod "491384e1-82a7-4067-975b-d50e947d4acf" (UID: "491384e1-82a7-4067-975b-d50e947d4acf"). InnerVolumeSpecName "kube-api-access-bmrd5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:07:19.200636 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:19.200611 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmrd5\" (UniqueName: \"kubernetes.io/projected/491384e1-82a7-4067-975b-d50e947d4acf-kube-api-access-bmrd5\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:07:19.739545 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:19.739508 2582 generic.go:356] "Generic (PLEG): container finished" podID="491384e1-82a7-4067-975b-d50e947d4acf" containerID="96200853f0a03e2a619aad1de9b2023198b9ffdcccb2182fe61eaf2357caa099" exitCode=0 Jul 09 02:07:19.739964 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:19.739554 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7s762" Jul 09 02:07:19.739964 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:19.739593 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7s762" event={"ID":"491384e1-82a7-4067-975b-d50e947d4acf","Type":"ContainerDied","Data":"96200853f0a03e2a619aad1de9b2023198b9ffdcccb2182fe61eaf2357caa099"} Jul 09 02:07:19.739964 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:19.739645 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7s762" event={"ID":"491384e1-82a7-4067-975b-d50e947d4acf","Type":"ContainerDied","Data":"2d36ccfd81397e7ef0a11b25affdd20de70ff77eb1d69ab8ce7f388b6ca4d5ec"} Jul 09 02:07:19.739964 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:19.739669 2582 scope.go:117] "RemoveContainer" containerID="96200853f0a03e2a619aad1de9b2023198b9ffdcccb2182fe61eaf2357caa099" Jul 09 02:07:19.748238 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:19.748221 2582 scope.go:117] "RemoveContainer" containerID="96200853f0a03e2a619aad1de9b2023198b9ffdcccb2182fe61eaf2357caa099" Jul 09 02:07:19.748500 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:07:19.748481 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96200853f0a03e2a619aad1de9b2023198b9ffdcccb2182fe61eaf2357caa099\": container with ID starting with 96200853f0a03e2a619aad1de9b2023198b9ffdcccb2182fe61eaf2357caa099 not found: ID does not exist" containerID="96200853f0a03e2a619aad1de9b2023198b9ffdcccb2182fe61eaf2357caa099" Jul 09 02:07:19.748546 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:19.748511 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96200853f0a03e2a619aad1de9b2023198b9ffdcccb2182fe61eaf2357caa099"} err="failed to get container status \"96200853f0a03e2a619aad1de9b2023198b9ffdcccb2182fe61eaf2357caa099\": rpc error: code = NotFound desc = could not find container \"96200853f0a03e2a619aad1de9b2023198b9ffdcccb2182fe61eaf2357caa099\": container with ID starting with 96200853f0a03e2a619aad1de9b2023198b9ffdcccb2182fe61eaf2357caa099 not found: ID does not exist" Jul 09 02:07:19.760467 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:19.760442 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7s762"] Jul 09 02:07:19.764113 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:19.764093 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7s762"] Jul 09 02:07:19.865262 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:19.865241 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="491384e1-82a7-4067-975b-d50e947d4acf" path="/var/lib/kubelet/pods/491384e1-82a7-4067-975b-d50e947d4acf/volumes" Jul 09 02:07:26.673578 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:26.673533 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-52mq2" Jul 09 02:07:26.674165 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:26.673600 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-52mq2" Jul 09 02:07:26.699086 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:26.699060 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-52mq2" Jul 09 02:07:26.792818 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:26.792790 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-52mq2" Jul 09 02:07:30.781652 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:30.781626 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh"] Jul 09 02:07:30.782059 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:30.781931 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="491384e1-82a7-4067-975b-d50e947d4acf" containerName="registry-server" Jul 09 02:07:30.782059 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:30.781943 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="491384e1-82a7-4067-975b-d50e947d4acf" containerName="registry-server" Jul 09 02:07:30.782059 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:30.782010 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="491384e1-82a7-4067-975b-d50e947d4acf" containerName="registry-server" Jul 09 02:07:30.786826 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:30.786796 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh" Jul 09 02:07:30.789825 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:30.789802 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xqpgc\"" Jul 09 02:07:30.793808 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:30.793784 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh"] Jul 09 02:07:30.880053 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:30.880025 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwm4b\" (UniqueName: \"kubernetes.io/projected/261f21fc-52be-4e8c-82ca-807f6ca6513d-kube-api-access-zwm4b\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh\" (UID: \"261f21fc-52be-4e8c-82ca-807f6ca6513d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh" Jul 09 02:07:30.880199 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:30.880071 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/261f21fc-52be-4e8c-82ca-807f6ca6513d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh\" (UID: \"261f21fc-52be-4e8c-82ca-807f6ca6513d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh" Jul 09 02:07:30.880199 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:30.880103 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/261f21fc-52be-4e8c-82ca-807f6ca6513d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh\" (UID: \"261f21fc-52be-4e8c-82ca-807f6ca6513d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh" Jul 09 02:07:30.981284 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:30.981256 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/261f21fc-52be-4e8c-82ca-807f6ca6513d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh\" (UID: \"261f21fc-52be-4e8c-82ca-807f6ca6513d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh" Jul 09 02:07:30.981451 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:30.981305 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwm4b\" (UniqueName: \"kubernetes.io/projected/261f21fc-52be-4e8c-82ca-807f6ca6513d-kube-api-access-zwm4b\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh\" (UID: \"261f21fc-52be-4e8c-82ca-807f6ca6513d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh" Jul 09 02:07:30.981451 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:30.981342 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/261f21fc-52be-4e8c-82ca-807f6ca6513d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh\" (UID: \"261f21fc-52be-4e8c-82ca-807f6ca6513d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh" Jul 09 02:07:30.981607 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:30.981590 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/261f21fc-52be-4e8c-82ca-807f6ca6513d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh\" (UID: \"261f21fc-52be-4e8c-82ca-807f6ca6513d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh" Jul 09 02:07:30.981651 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:30.981613 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/261f21fc-52be-4e8c-82ca-807f6ca6513d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh\" (UID: \"261f21fc-52be-4e8c-82ca-807f6ca6513d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh" Jul 09 02:07:30.991475 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:30.991449 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwm4b\" (UniqueName: \"kubernetes.io/projected/261f21fc-52be-4e8c-82ca-807f6ca6513d-kube-api-access-zwm4b\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh\" (UID: \"261f21fc-52be-4e8c-82ca-807f6ca6513d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh" Jul 09 02:07:31.096490 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.096399 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh" Jul 09 02:07:31.241711 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.241685 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh"] Jul 09 02:07:31.244377 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:07:31.244353 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod261f21fc_52be_4e8c_82ca_807f6ca6513d.slice/crio-69354e7337942ccaacc313ad6f64cee179b5b256302d326a7b3cf7bfd92e849e WatchSource:0}: Error finding container 69354e7337942ccaacc313ad6f64cee179b5b256302d326a7b3cf7bfd92e849e: Status 404 returned error can't find the container with id 69354e7337942ccaacc313ad6f64cee179b5b256302d326a7b3cf7bfd92e849e Jul 09 02:07:31.377842 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.377809 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2"] Jul 09 02:07:31.381064 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.381046 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2" Jul 09 02:07:31.391469 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.391442 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2"] Jul 09 02:07:31.485709 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.485685 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b19afe3c-311b-4c34-94ef-975696e39caf-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2\" (UID: \"b19afe3c-311b-4c34-94ef-975696e39caf\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2" Jul 09 02:07:31.485834 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.485737 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4826x\" (UniqueName: \"kubernetes.io/projected/b19afe3c-311b-4c34-94ef-975696e39caf-kube-api-access-4826x\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2\" (UID: \"b19afe3c-311b-4c34-94ef-975696e39caf\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2" Jul 09 02:07:31.485834 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.485770 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b19afe3c-311b-4c34-94ef-975696e39caf-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2\" (UID: \"b19afe3c-311b-4c34-94ef-975696e39caf\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2" Jul 09 02:07:31.587043 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.586758 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b19afe3c-311b-4c34-94ef-975696e39caf-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2\" (UID: \"b19afe3c-311b-4c34-94ef-975696e39caf\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2" Jul 09 02:07:31.587043 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.586912 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4826x\" (UniqueName: \"kubernetes.io/projected/b19afe3c-311b-4c34-94ef-975696e39caf-kube-api-access-4826x\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2\" (UID: \"b19afe3c-311b-4c34-94ef-975696e39caf\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2" Jul 09 02:07:31.587043 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.586989 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b19afe3c-311b-4c34-94ef-975696e39caf-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2\" (UID: \"b19afe3c-311b-4c34-94ef-975696e39caf\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2" Jul 09 02:07:31.587258 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.587216 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b19afe3c-311b-4c34-94ef-975696e39caf-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2\" (UID: \"b19afe3c-311b-4c34-94ef-975696e39caf\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2" Jul 09 02:07:31.591418 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.587658 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b19afe3c-311b-4c34-94ef-975696e39caf-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2\" (UID: \"b19afe3c-311b-4c34-94ef-975696e39caf\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2" Jul 09 02:07:31.597752 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.597726 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4826x\" (UniqueName: \"kubernetes.io/projected/b19afe3c-311b-4c34-94ef-975696e39caf-kube-api-access-4826x\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2\" (UID: \"b19afe3c-311b-4c34-94ef-975696e39caf\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2" Jul 09 02:07:31.722116 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.722090 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2" Jul 09 02:07:31.793600 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.793570 2582 generic.go:356] "Generic (PLEG): container finished" podID="261f21fc-52be-4e8c-82ca-807f6ca6513d" containerID="2f16221bf4ce466d87871768cdfc1e777fab7ec3ec687aee8393a8be58e521c2" exitCode=0 Jul 09 02:07:31.794053 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.793611 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh" event={"ID":"261f21fc-52be-4e8c-82ca-807f6ca6513d","Type":"ContainerDied","Data":"2f16221bf4ce466d87871768cdfc1e777fab7ec3ec687aee8393a8be58e521c2"} Jul 09 02:07:31.794053 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.793671 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh" event={"ID":"261f21fc-52be-4e8c-82ca-807f6ca6513d","Type":"ContainerStarted","Data":"69354e7337942ccaacc313ad6f64cee179b5b256302d326a7b3cf7bfd92e849e"} Jul 09 02:07:31.851797 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.851773 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2"] Jul 09 02:07:31.854026 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:07:31.854001 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb19afe3c_311b_4c34_94ef_975696e39caf.slice/crio-72809ebe5110d99e58a02692baae7c92e70f767adac7986ee4eb50fe2d9f0a10 WatchSource:0}: Error finding container 72809ebe5110d99e58a02692baae7c92e70f767adac7986ee4eb50fe2d9f0a10: Status 404 returned error can't find the container with id 72809ebe5110d99e58a02692baae7c92e70f767adac7986ee4eb50fe2d9f0a10 Jul 09 02:07:31.980447 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.980386 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf"] Jul 09 02:07:31.983792 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.983772 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf" Jul 09 02:07:31.992056 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:31.992036 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf"] Jul 09 02:07:32.090794 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.090759 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b859e0af-2e73-42e4-a3f4-36cc346aac92-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf\" (UID: \"b859e0af-2e73-42e4-a3f4-36cc346aac92\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf" Jul 09 02:07:32.090979 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.090814 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b859e0af-2e73-42e4-a3f4-36cc346aac92-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf\" (UID: \"b859e0af-2e73-42e4-a3f4-36cc346aac92\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf" Jul 09 02:07:32.090979 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.090839 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvh7d\" (UniqueName: \"kubernetes.io/projected/b859e0af-2e73-42e4-a3f4-36cc346aac92-kube-api-access-wvh7d\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf\" (UID: \"b859e0af-2e73-42e4-a3f4-36cc346aac92\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf" Jul 09 02:07:32.191479 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.191447 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvh7d\" (UniqueName: \"kubernetes.io/projected/b859e0af-2e73-42e4-a3f4-36cc346aac92-kube-api-access-wvh7d\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf\" (UID: \"b859e0af-2e73-42e4-a3f4-36cc346aac92\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf" Jul 09 02:07:32.191624 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.191502 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b859e0af-2e73-42e4-a3f4-36cc346aac92-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf\" (UID: \"b859e0af-2e73-42e4-a3f4-36cc346aac92\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf" Jul 09 02:07:32.191624 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.191539 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b859e0af-2e73-42e4-a3f4-36cc346aac92-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf\" (UID: \"b859e0af-2e73-42e4-a3f4-36cc346aac92\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf" Jul 09 02:07:32.191897 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.191857 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b859e0af-2e73-42e4-a3f4-36cc346aac92-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf\" (UID: \"b859e0af-2e73-42e4-a3f4-36cc346aac92\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf" Jul 09 02:07:32.191942 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.191902 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b859e0af-2e73-42e4-a3f4-36cc346aac92-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf\" (UID: \"b859e0af-2e73-42e4-a3f4-36cc346aac92\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf" Jul 09 02:07:32.201267 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.201248 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvh7d\" (UniqueName: \"kubernetes.io/projected/b859e0af-2e73-42e4-a3f4-36cc346aac92-kube-api-access-wvh7d\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf\" (UID: \"b859e0af-2e73-42e4-a3f4-36cc346aac92\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf" Jul 09 02:07:32.319475 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.319403 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf" Jul 09 02:07:32.382761 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.382735 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l"] Jul 09 02:07:32.387191 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.387168 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l" Jul 09 02:07:32.395523 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.395500 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l"] Jul 09 02:07:32.453115 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.453079 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf"] Jul 09 02:07:32.484998 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:07:32.484973 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb859e0af_2e73_42e4_a3f4_36cc346aac92.slice/crio-f81080d0f831e4a6112afb309f9f30d65ab169393ea2229bba45fa617b1e4734 WatchSource:0}: Error finding container f81080d0f831e4a6112afb309f9f30d65ab169393ea2229bba45fa617b1e4734: Status 404 returned error can't find the container with id f81080d0f831e4a6112afb309f9f30d65ab169393ea2229bba45fa617b1e4734 Jul 09 02:07:32.493653 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.493628 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34076dc0-01aa-4322-b5a4-ccf17bb46fad-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l\" (UID: \"34076dc0-01aa-4322-b5a4-ccf17bb46fad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l" Jul 09 02:07:32.493741 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.493728 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34076dc0-01aa-4322-b5a4-ccf17bb46fad-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l\" (UID: \"34076dc0-01aa-4322-b5a4-ccf17bb46fad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l" Jul 09 02:07:32.493813 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.493796 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnwb6\" (UniqueName: \"kubernetes.io/projected/34076dc0-01aa-4322-b5a4-ccf17bb46fad-kube-api-access-tnwb6\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l\" (UID: \"34076dc0-01aa-4322-b5a4-ccf17bb46fad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l" Jul 09 02:07:32.594732 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.594713 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnwb6\" (UniqueName: \"kubernetes.io/projected/34076dc0-01aa-4322-b5a4-ccf17bb46fad-kube-api-access-tnwb6\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l\" (UID: \"34076dc0-01aa-4322-b5a4-ccf17bb46fad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l" Jul 09 02:07:32.594837 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.594748 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34076dc0-01aa-4322-b5a4-ccf17bb46fad-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l\" (UID: \"34076dc0-01aa-4322-b5a4-ccf17bb46fad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l" Jul 09 02:07:32.594837 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.594789 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34076dc0-01aa-4322-b5a4-ccf17bb46fad-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l\" (UID: \"34076dc0-01aa-4322-b5a4-ccf17bb46fad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l" Jul 09 02:07:32.595101 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.595086 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34076dc0-01aa-4322-b5a4-ccf17bb46fad-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l\" (UID: \"34076dc0-01aa-4322-b5a4-ccf17bb46fad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l" Jul 09 02:07:32.595154 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.595128 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34076dc0-01aa-4322-b5a4-ccf17bb46fad-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l\" (UID: \"34076dc0-01aa-4322-b5a4-ccf17bb46fad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l" Jul 09 02:07:32.604277 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.604255 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnwb6\" (UniqueName: \"kubernetes.io/projected/34076dc0-01aa-4322-b5a4-ccf17bb46fad-kube-api-access-tnwb6\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l\" (UID: \"34076dc0-01aa-4322-b5a4-ccf17bb46fad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l" Jul 09 02:07:32.697304 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.697284 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l" Jul 09 02:07:32.799967 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.799932 2582 generic.go:356] "Generic (PLEG): container finished" podID="261f21fc-52be-4e8c-82ca-807f6ca6513d" containerID="d329ed6a6320f4fb556046b32c16e70e16fe13de77e69d5a025ab72bcf43a49d" exitCode=0 Jul 09 02:07:32.800396 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.800006 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh" event={"ID":"261f21fc-52be-4e8c-82ca-807f6ca6513d","Type":"ContainerDied","Data":"d329ed6a6320f4fb556046b32c16e70e16fe13de77e69d5a025ab72bcf43a49d"} Jul 09 02:07:32.801951 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.801929 2582 generic.go:356] "Generic (PLEG): container finished" podID="b19afe3c-311b-4c34-94ef-975696e39caf" containerID="ed587ba668bb11a7763641d00e12dfe4bc262be7235fda3bf7c481f0933d1b05" exitCode=0 Jul 09 02:07:32.802074 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.802013 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2" event={"ID":"b19afe3c-311b-4c34-94ef-975696e39caf","Type":"ContainerDied","Data":"ed587ba668bb11a7763641d00e12dfe4bc262be7235fda3bf7c481f0933d1b05"} Jul 09 02:07:32.802074 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.802063 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2" event={"ID":"b19afe3c-311b-4c34-94ef-975696e39caf","Type":"ContainerStarted","Data":"72809ebe5110d99e58a02692baae7c92e70f767adac7986ee4eb50fe2d9f0a10"} Jul 09 02:07:32.803819 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.803798 2582 generic.go:356] "Generic (PLEG): container finished" podID="b859e0af-2e73-42e4-a3f4-36cc346aac92" containerID="b8514d50334f00518e65d351141bb01729e8ba3d73b5f56e9bfa42e6b2d7b63c" exitCode=0 Jul 09 02:07:32.803935 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.803894 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf" event={"ID":"b859e0af-2e73-42e4-a3f4-36cc346aac92","Type":"ContainerDied","Data":"b8514d50334f00518e65d351141bb01729e8ba3d73b5f56e9bfa42e6b2d7b63c"} Jul 09 02:07:32.803997 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.803944 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf" event={"ID":"b859e0af-2e73-42e4-a3f4-36cc346aac92","Type":"ContainerStarted","Data":"f81080d0f831e4a6112afb309f9f30d65ab169393ea2229bba45fa617b1e4734"} Jul 09 02:07:32.823506 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:32.823485 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l"] Jul 09 02:07:32.826855 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:07:32.826834 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34076dc0_01aa_4322_b5a4_ccf17bb46fad.slice/crio-9b13bea8442b335df9103a6f0e3968e0a72be94bbc16aa9dcbb3396bdb75f462 WatchSource:0}: Error finding container 9b13bea8442b335df9103a6f0e3968e0a72be94bbc16aa9dcbb3396bdb75f462: Status 404 returned error can't find the container with id 9b13bea8442b335df9103a6f0e3968e0a72be94bbc16aa9dcbb3396bdb75f462 Jul 09 02:07:33.810411 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:33.810390 2582 generic.go:356] "Generic (PLEG): container finished" podID="261f21fc-52be-4e8c-82ca-807f6ca6513d" containerID="6972c9c5ba232691a894f5853fe8745663f388df85b4e3ef2310109732df26e4" exitCode=0 Jul 09 02:07:33.810724 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:33.810474 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh" event={"ID":"261f21fc-52be-4e8c-82ca-807f6ca6513d","Type":"ContainerDied","Data":"6972c9c5ba232691a894f5853fe8745663f388df85b4e3ef2310109732df26e4"} Jul 09 02:07:33.812031 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:33.812015 2582 generic.go:356] "Generic (PLEG): container finished" podID="b19afe3c-311b-4c34-94ef-975696e39caf" containerID="2ed80f62d2def9162fa5df6840ddfa1865403c041e716b24ccc8df1bf861111a" exitCode=0 Jul 09 02:07:33.812114 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:33.812097 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2" event={"ID":"b19afe3c-311b-4c34-94ef-975696e39caf","Type":"ContainerDied","Data":"2ed80f62d2def9162fa5df6840ddfa1865403c041e716b24ccc8df1bf861111a"} Jul 09 02:07:33.813697 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:33.813681 2582 generic.go:356] "Generic (PLEG): container finished" podID="b859e0af-2e73-42e4-a3f4-36cc346aac92" containerID="2b96d77babb6dd673e769ad54825c097c40139ad7988257ee1c322723d991317" exitCode=0 Jul 09 02:07:33.813784 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:33.813759 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf" event={"ID":"b859e0af-2e73-42e4-a3f4-36cc346aac92","Type":"ContainerDied","Data":"2b96d77babb6dd673e769ad54825c097c40139ad7988257ee1c322723d991317"} Jul 09 02:07:33.815219 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:33.815202 2582 generic.go:356] "Generic (PLEG): container finished" podID="34076dc0-01aa-4322-b5a4-ccf17bb46fad" containerID="7b7512ce47728f26d7aad6dba7c230e863f5e1d91536a5ed03bef1a09e883d64" exitCode=0 Jul 09 02:07:33.815294 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:33.815230 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l" event={"ID":"34076dc0-01aa-4322-b5a4-ccf17bb46fad","Type":"ContainerDied","Data":"7b7512ce47728f26d7aad6dba7c230e863f5e1d91536a5ed03bef1a09e883d64"} Jul 09 02:07:33.815294 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:33.815265 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l" event={"ID":"34076dc0-01aa-4322-b5a4-ccf17bb46fad","Type":"ContainerStarted","Data":"9b13bea8442b335df9103a6f0e3968e0a72be94bbc16aa9dcbb3396bdb75f462"} Jul 09 02:07:34.710417 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:34.710373 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:07:34.811620 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:34.811576 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:07:34.821626 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:34.821596 2582 generic.go:356] "Generic (PLEG): container finished" podID="b859e0af-2e73-42e4-a3f4-36cc346aac92" containerID="50cf0b70752820acde5d8b25253cdffd943f6da7b32f8df428e91f0653d29b14" exitCode=0 Jul 09 02:07:34.821728 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:34.821672 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf" event={"ID":"b859e0af-2e73-42e4-a3f4-36cc346aac92","Type":"ContainerDied","Data":"50cf0b70752820acde5d8b25253cdffd943f6da7b32f8df428e91f0653d29b14"} Jul 09 02:07:34.823316 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:34.823295 2582 generic.go:356] "Generic (PLEG): container finished" podID="34076dc0-01aa-4322-b5a4-ccf17bb46fad" containerID="366f5cf04b2329327a4bffe91e54d3a2e62356972db0e47c546a22ebbb422112" exitCode=0 Jul 09 02:07:34.823423 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:34.823379 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l" event={"ID":"34076dc0-01aa-4322-b5a4-ccf17bb46fad","Type":"ContainerDied","Data":"366f5cf04b2329327a4bffe91e54d3a2e62356972db0e47c546a22ebbb422112"} Jul 09 02:07:34.825417 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:34.825399 2582 generic.go:356] "Generic (PLEG): container finished" podID="b19afe3c-311b-4c34-94ef-975696e39caf" containerID="5299c87819624c5207869f356476f929649997be4f402ee384f1381885c172b2" exitCode=0 Jul 09 02:07:34.825516 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:34.825480 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2" event={"ID":"b19afe3c-311b-4c34-94ef-975696e39caf","Type":"ContainerDied","Data":"5299c87819624c5207869f356476f929649997be4f402ee384f1381885c172b2"} Jul 09 02:07:34.955116 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:34.955094 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh" Jul 09 02:07:35.015813 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:35.015781 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/261f21fc-52be-4e8c-82ca-807f6ca6513d-bundle\") pod \"261f21fc-52be-4e8c-82ca-807f6ca6513d\" (UID: \"261f21fc-52be-4e8c-82ca-807f6ca6513d\") " Jul 09 02:07:35.015949 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:35.015840 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/261f21fc-52be-4e8c-82ca-807f6ca6513d-util\") pod \"261f21fc-52be-4e8c-82ca-807f6ca6513d\" (UID: \"261f21fc-52be-4e8c-82ca-807f6ca6513d\") " Jul 09 02:07:35.015949 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:35.015923 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwm4b\" (UniqueName: \"kubernetes.io/projected/261f21fc-52be-4e8c-82ca-807f6ca6513d-kube-api-access-zwm4b\") pod \"261f21fc-52be-4e8c-82ca-807f6ca6513d\" (UID: \"261f21fc-52be-4e8c-82ca-807f6ca6513d\") " Jul 09 02:07:35.016457 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:35.016420 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/261f21fc-52be-4e8c-82ca-807f6ca6513d-bundle" (OuterVolumeSpecName: "bundle") pod "261f21fc-52be-4e8c-82ca-807f6ca6513d" (UID: "261f21fc-52be-4e8c-82ca-807f6ca6513d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:07:35.017842 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:35.017817 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/261f21fc-52be-4e8c-82ca-807f6ca6513d-kube-api-access-zwm4b" (OuterVolumeSpecName: "kube-api-access-zwm4b") pod "261f21fc-52be-4e8c-82ca-807f6ca6513d" (UID: "261f21fc-52be-4e8c-82ca-807f6ca6513d"). InnerVolumeSpecName "kube-api-access-zwm4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:07:35.021636 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:35.021613 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/261f21fc-52be-4e8c-82ca-807f6ca6513d-util" (OuterVolumeSpecName: "util") pod "261f21fc-52be-4e8c-82ca-807f6ca6513d" (UID: "261f21fc-52be-4e8c-82ca-807f6ca6513d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:07:35.117451 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:35.117415 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/261f21fc-52be-4e8c-82ca-807f6ca6513d-bundle\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:07:35.117451 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:35.117444 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/261f21fc-52be-4e8c-82ca-807f6ca6513d-util\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:07:35.117451 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:35.117456 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwm4b\" (UniqueName: \"kubernetes.io/projected/261f21fc-52be-4e8c-82ca-807f6ca6513d-kube-api-access-zwm4b\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:07:35.832109 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:35.832075 2582 generic.go:356] "Generic (PLEG): container finished" podID="34076dc0-01aa-4322-b5a4-ccf17bb46fad" containerID="8811f3a4435e4955e15262ad6f762a8643aba98a5d9207731c4921cc78b51219" exitCode=0 Jul 09 02:07:35.832535 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:35.832150 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l" event={"ID":"34076dc0-01aa-4322-b5a4-ccf17bb46fad","Type":"ContainerDied","Data":"8811f3a4435e4955e15262ad6f762a8643aba98a5d9207731c4921cc78b51219"} Jul 09 02:07:35.834015 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:35.833992 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh" event={"ID":"261f21fc-52be-4e8c-82ca-807f6ca6513d","Type":"ContainerDied","Data":"69354e7337942ccaacc313ad6f64cee179b5b256302d326a7b3cf7bfd92e849e"} Jul 09 02:07:35.834119 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:35.834040 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69354e7337942ccaacc313ad6f64cee179b5b256302d326a7b3cf7bfd92e849e" Jul 09 02:07:35.834119 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:35.834002 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh" Jul 09 02:07:35.992513 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:35.992495 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf" Jul 09 02:07:35.994494 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:35.994477 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2" Jul 09 02:07:36.126039 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.125958 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvh7d\" (UniqueName: \"kubernetes.io/projected/b859e0af-2e73-42e4-a3f4-36cc346aac92-kube-api-access-wvh7d\") pod \"b859e0af-2e73-42e4-a3f4-36cc346aac92\" (UID: \"b859e0af-2e73-42e4-a3f4-36cc346aac92\") " Jul 09 02:07:36.126039 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.126005 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b859e0af-2e73-42e4-a3f4-36cc346aac92-util\") pod \"b859e0af-2e73-42e4-a3f4-36cc346aac92\" (UID: \"b859e0af-2e73-42e4-a3f4-36cc346aac92\") " Jul 09 02:07:36.126039 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.126037 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b859e0af-2e73-42e4-a3f4-36cc346aac92-bundle\") pod \"b859e0af-2e73-42e4-a3f4-36cc346aac92\" (UID: \"b859e0af-2e73-42e4-a3f4-36cc346aac92\") " Jul 09 02:07:36.126308 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.126085 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b19afe3c-311b-4c34-94ef-975696e39caf-util\") pod \"b19afe3c-311b-4c34-94ef-975696e39caf\" (UID: \"b19afe3c-311b-4c34-94ef-975696e39caf\") " Jul 09 02:07:36.126308 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.126125 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b19afe3c-311b-4c34-94ef-975696e39caf-bundle\") pod \"b19afe3c-311b-4c34-94ef-975696e39caf\" (UID: \"b19afe3c-311b-4c34-94ef-975696e39caf\") " Jul 09 02:07:36.126308 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.126151 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4826x\" (UniqueName: \"kubernetes.io/projected/b19afe3c-311b-4c34-94ef-975696e39caf-kube-api-access-4826x\") pod \"b19afe3c-311b-4c34-94ef-975696e39caf\" (UID: \"b19afe3c-311b-4c34-94ef-975696e39caf\") " Jul 09 02:07:36.126660 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.126632 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b19afe3c-311b-4c34-94ef-975696e39caf-bundle" (OuterVolumeSpecName: "bundle") pod "b19afe3c-311b-4c34-94ef-975696e39caf" (UID: "b19afe3c-311b-4c34-94ef-975696e39caf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:07:36.126730 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.126665 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b859e0af-2e73-42e4-a3f4-36cc346aac92-bundle" (OuterVolumeSpecName: "bundle") pod "b859e0af-2e73-42e4-a3f4-36cc346aac92" (UID: "b859e0af-2e73-42e4-a3f4-36cc346aac92"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:07:36.128280 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.128251 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b19afe3c-311b-4c34-94ef-975696e39caf-kube-api-access-4826x" (OuterVolumeSpecName: "kube-api-access-4826x") pod "b19afe3c-311b-4c34-94ef-975696e39caf" (UID: "b19afe3c-311b-4c34-94ef-975696e39caf"). InnerVolumeSpecName "kube-api-access-4826x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:07:36.128437 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.128418 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b859e0af-2e73-42e4-a3f4-36cc346aac92-kube-api-access-wvh7d" (OuterVolumeSpecName: "kube-api-access-wvh7d") pod "b859e0af-2e73-42e4-a3f4-36cc346aac92" (UID: "b859e0af-2e73-42e4-a3f4-36cc346aac92"). InnerVolumeSpecName "kube-api-access-wvh7d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:07:36.132351 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.132319 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b859e0af-2e73-42e4-a3f4-36cc346aac92-util" (OuterVolumeSpecName: "util") pod "b859e0af-2e73-42e4-a3f4-36cc346aac92" (UID: "b859e0af-2e73-42e4-a3f4-36cc346aac92"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:07:36.134340 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.134316 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b19afe3c-311b-4c34-94ef-975696e39caf-util" (OuterVolumeSpecName: "util") pod "b19afe3c-311b-4c34-94ef-975696e39caf" (UID: "b19afe3c-311b-4c34-94ef-975696e39caf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:07:36.226809 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.226776 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvh7d\" (UniqueName: \"kubernetes.io/projected/b859e0af-2e73-42e4-a3f4-36cc346aac92-kube-api-access-wvh7d\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:07:36.226809 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.226807 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b859e0af-2e73-42e4-a3f4-36cc346aac92-util\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:07:36.226809 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.226817 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b859e0af-2e73-42e4-a3f4-36cc346aac92-bundle\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:07:36.227009 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.226825 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b19afe3c-311b-4c34-94ef-975696e39caf-util\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:07:36.227009 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.226833 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b19afe3c-311b-4c34-94ef-975696e39caf-bundle\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:07:36.227009 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.226840 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4826x\" (UniqueName: \"kubernetes.io/projected/b19afe3c-311b-4c34-94ef-975696e39caf-kube-api-access-4826x\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:07:36.841169 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.841141 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2" Jul 09 02:07:36.841169 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.841148 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2" event={"ID":"b19afe3c-311b-4c34-94ef-975696e39caf","Type":"ContainerDied","Data":"72809ebe5110d99e58a02692baae7c92e70f767adac7986ee4eb50fe2d9f0a10"} Jul 09 02:07:36.841601 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.841193 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72809ebe5110d99e58a02692baae7c92e70f767adac7986ee4eb50fe2d9f0a10" Jul 09 02:07:36.843008 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.842991 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf" Jul 09 02:07:36.843119 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.843018 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf" event={"ID":"b859e0af-2e73-42e4-a3f4-36cc346aac92","Type":"ContainerDied","Data":"f81080d0f831e4a6112afb309f9f30d65ab169393ea2229bba45fa617b1e4734"} Jul 09 02:07:36.843119 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.843052 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f81080d0f831e4a6112afb309f9f30d65ab169393ea2229bba45fa617b1e4734" Jul 09 02:07:36.969442 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:36.969415 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l" Jul 09 02:07:37.033367 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:37.033332 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34076dc0-01aa-4322-b5a4-ccf17bb46fad-util\") pod \"34076dc0-01aa-4322-b5a4-ccf17bb46fad\" (UID: \"34076dc0-01aa-4322-b5a4-ccf17bb46fad\") " Jul 09 02:07:37.033498 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:37.033376 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34076dc0-01aa-4322-b5a4-ccf17bb46fad-bundle\") pod \"34076dc0-01aa-4322-b5a4-ccf17bb46fad\" (UID: \"34076dc0-01aa-4322-b5a4-ccf17bb46fad\") " Jul 09 02:07:37.033498 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:37.033406 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnwb6\" (UniqueName: \"kubernetes.io/projected/34076dc0-01aa-4322-b5a4-ccf17bb46fad-kube-api-access-tnwb6\") pod \"34076dc0-01aa-4322-b5a4-ccf17bb46fad\" (UID: \"34076dc0-01aa-4322-b5a4-ccf17bb46fad\") " Jul 09 02:07:37.033819 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:37.033796 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34076dc0-01aa-4322-b5a4-ccf17bb46fad-bundle" (OuterVolumeSpecName: "bundle") pod "34076dc0-01aa-4322-b5a4-ccf17bb46fad" (UID: "34076dc0-01aa-4322-b5a4-ccf17bb46fad"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:07:37.035528 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:37.035507 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34076dc0-01aa-4322-b5a4-ccf17bb46fad-kube-api-access-tnwb6" (OuterVolumeSpecName: "kube-api-access-tnwb6") pod "34076dc0-01aa-4322-b5a4-ccf17bb46fad" (UID: "34076dc0-01aa-4322-b5a4-ccf17bb46fad"). InnerVolumeSpecName "kube-api-access-tnwb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:07:37.038766 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:37.038746 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34076dc0-01aa-4322-b5a4-ccf17bb46fad-util" (OuterVolumeSpecName: "util") pod "34076dc0-01aa-4322-b5a4-ccf17bb46fad" (UID: "34076dc0-01aa-4322-b5a4-ccf17bb46fad"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:07:37.134190 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:37.134111 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34076dc0-01aa-4322-b5a4-ccf17bb46fad-util\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:07:37.134190 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:37.134140 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34076dc0-01aa-4322-b5a4-ccf17bb46fad-bundle\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:07:37.134190 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:37.134151 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnwb6\" (UniqueName: \"kubernetes.io/projected/34076dc0-01aa-4322-b5a4-ccf17bb46fad-kube-api-access-tnwb6\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:07:37.851363 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:37.851329 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l" event={"ID":"34076dc0-01aa-4322-b5a4-ccf17bb46fad","Type":"ContainerDied","Data":"9b13bea8442b335df9103a6f0e3968e0a72be94bbc16aa9dcbb3396bdb75f462"} Jul 09 02:07:37.851765 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:37.851370 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l" Jul 09 02:07:37.851765 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:37.851405 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b13bea8442b335df9103a6f0e3968e0a72be94bbc16aa9dcbb3396bdb75f462" Jul 09 02:07:44.502916 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:44.502881 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:07:45.893418 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:45.893382 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:07:45.951056 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:45.951013 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:07:46.012093 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:46.012060 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:07:47.159342 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159310 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lngv4"] Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159588 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b19afe3c-311b-4c34-94ef-975696e39caf" containerName="pull" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159599 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b19afe3c-311b-4c34-94ef-975696e39caf" containerName="pull" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159609 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b19afe3c-311b-4c34-94ef-975696e39caf" containerName="extract" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159614 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b19afe3c-311b-4c34-94ef-975696e39caf" containerName="extract" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159631 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b859e0af-2e73-42e4-a3f4-36cc346aac92" containerName="extract" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159636 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b859e0af-2e73-42e4-a3f4-36cc346aac92" containerName="extract" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159643 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="261f21fc-52be-4e8c-82ca-807f6ca6513d" containerName="util" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159648 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="261f21fc-52be-4e8c-82ca-807f6ca6513d" containerName="util" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159655 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34076dc0-01aa-4322-b5a4-ccf17bb46fad" containerName="extract" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159660 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="34076dc0-01aa-4322-b5a4-ccf17bb46fad" containerName="extract" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159670 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34076dc0-01aa-4322-b5a4-ccf17bb46fad" containerName="util" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159675 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="34076dc0-01aa-4322-b5a4-ccf17bb46fad" containerName="util" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159682 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b19afe3c-311b-4c34-94ef-975696e39caf" containerName="util" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159688 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b19afe3c-311b-4c34-94ef-975696e39caf" containerName="util" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159693 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="261f21fc-52be-4e8c-82ca-807f6ca6513d" containerName="pull" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159698 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="261f21fc-52be-4e8c-82ca-807f6ca6513d" containerName="pull" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159704 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34076dc0-01aa-4322-b5a4-ccf17bb46fad" containerName="pull" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159708 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="34076dc0-01aa-4322-b5a4-ccf17bb46fad" containerName="pull" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159715 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b859e0af-2e73-42e4-a3f4-36cc346aac92" containerName="util" Jul 09 02:07:47.159714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159721 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b859e0af-2e73-42e4-a3f4-36cc346aac92" containerName="util" Jul 09 02:07:47.160284 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159729 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="261f21fc-52be-4e8c-82ca-807f6ca6513d" containerName="extract" Jul 09 02:07:47.160284 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159734 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="261f21fc-52be-4e8c-82ca-807f6ca6513d" containerName="extract" Jul 09 02:07:47.160284 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159740 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b859e0af-2e73-42e4-a3f4-36cc346aac92" containerName="pull" Jul 09 02:07:47.160284 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159745 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b859e0af-2e73-42e4-a3f4-36cc346aac92" containerName="pull" Jul 09 02:07:47.160284 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159791 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="261f21fc-52be-4e8c-82ca-807f6ca6513d" containerName="extract" Jul 09 02:07:47.160284 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159798 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b19afe3c-311b-4c34-94ef-975696e39caf" containerName="extract" Jul 09 02:07:47.160284 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159804 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b859e0af-2e73-42e4-a3f4-36cc346aac92" containerName="extract" Jul 09 02:07:47.160284 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.159811 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="34076dc0-01aa-4322-b5a4-ccf17bb46fad" containerName="extract" Jul 09 02:07:47.163715 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.163696 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lngv4" Jul 09 02:07:47.166993 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.166975 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-k24pj\"" Jul 09 02:07:47.167072 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.167029 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 09 02:07:47.192336 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.192308 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lngv4"] Jul 09 02:07:47.310201 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.310167 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sncv\" (UniqueName: \"kubernetes.io/projected/ca1cefe0-a5f8-4793-a7c6-ff21c7996095-kube-api-access-6sncv\") pod \"dns-operator-controller-manager-864dd78c4c-lngv4\" (UID: \"ca1cefe0-a5f8-4793-a7c6-ff21c7996095\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lngv4" Jul 09 02:07:47.410595 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.410511 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sncv\" (UniqueName: \"kubernetes.io/projected/ca1cefe0-a5f8-4793-a7c6-ff21c7996095-kube-api-access-6sncv\") pod \"dns-operator-controller-manager-864dd78c4c-lngv4\" (UID: \"ca1cefe0-a5f8-4793-a7c6-ff21c7996095\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lngv4" Jul 09 02:07:47.428555 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.428525 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sncv\" (UniqueName: \"kubernetes.io/projected/ca1cefe0-a5f8-4793-a7c6-ff21c7996095-kube-api-access-6sncv\") pod \"dns-operator-controller-manager-864dd78c4c-lngv4\" (UID: \"ca1cefe0-a5f8-4793-a7c6-ff21c7996095\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lngv4" Jul 09 02:07:47.473659 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.473624 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lngv4" Jul 09 02:07:47.612176 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.612156 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lngv4"] Jul 09 02:07:47.614427 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:07:47.614402 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca1cefe0_a5f8_4793_a7c6_ff21c7996095.slice/crio-c86ac27fd0da2681af95189d12de548431f8c74135ac73916e983fb04f85a1df WatchSource:0}: Error finding container c86ac27fd0da2681af95189d12de548431f8c74135ac73916e983fb04f85a1df: Status 404 returned error can't find the container with id c86ac27fd0da2681af95189d12de548431f8c74135ac73916e983fb04f85a1df Jul 09 02:07:47.894091 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:47.894021 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lngv4" event={"ID":"ca1cefe0-a5f8-4793-a7c6-ff21c7996095","Type":"ContainerStarted","Data":"c86ac27fd0da2681af95189d12de548431f8c74135ac73916e983fb04f85a1df"} Jul 09 02:07:48.105274 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:48.105235 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:07:49.905454 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:49.905421 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lngv4" event={"ID":"ca1cefe0-a5f8-4793-a7c6-ff21c7996095","Type":"ContainerStarted","Data":"7f78b47f02ec24299ea8319d5806b5c0d354ea91f05c02b60ba3ce223a33ac99"} Jul 09 02:07:49.905767 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:49.905543 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lngv4" Jul 09 02:07:49.925930 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:49.925889 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lngv4" podStartSLOduration=0.754853612 podStartE2EDuration="2.925861444s" totalImagesPullingTime="2.171007832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:07:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:07:49.925027177 +0000 UTC m=+490.623080989" watchObservedRunningTime="2026-07-09 02:07:49.925861444 +0000 UTC m=+490.623915257" Jul 09 02:07:51.148186 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:51.148155 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:07:52.468772 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:52.468742 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv"] Jul 09 02:07:52.475136 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:52.475115 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" Jul 09 02:07:52.478044 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:52.478028 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f78gw\"" Jul 09 02:07:52.486834 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:52.486811 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv"] Jul 09 02:07:52.549726 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:52.549705 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvbpv\" (UniqueName: \"kubernetes.io/projected/6b0f85f3-9265-4926-a1de-653dc6203a0e-kube-api-access-cvbpv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mc7qv\" (UID: \"6b0f85f3-9265-4926-a1de-653dc6203a0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" Jul 09 02:07:52.549840 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:52.549735 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b0f85f3-9265-4926-a1de-653dc6203a0e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mc7qv\" (UID: \"6b0f85f3-9265-4926-a1de-653dc6203a0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" Jul 09 02:07:52.603626 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:52.603601 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:07:52.650939 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:52.650905 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvbpv\" (UniqueName: \"kubernetes.io/projected/6b0f85f3-9265-4926-a1de-653dc6203a0e-kube-api-access-cvbpv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mc7qv\" (UID: \"6b0f85f3-9265-4926-a1de-653dc6203a0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" Jul 09 02:07:52.651045 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:52.650946 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b0f85f3-9265-4926-a1de-653dc6203a0e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mc7qv\" (UID: \"6b0f85f3-9265-4926-a1de-653dc6203a0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" Jul 09 02:07:52.651310 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:52.651297 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b0f85f3-9265-4926-a1de-653dc6203a0e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mc7qv\" (UID: \"6b0f85f3-9265-4926-a1de-653dc6203a0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" Jul 09 02:07:52.663790 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:52.663769 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvbpv\" (UniqueName: \"kubernetes.io/projected/6b0f85f3-9265-4926-a1de-653dc6203a0e-kube-api-access-cvbpv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mc7qv\" (UID: \"6b0f85f3-9265-4926-a1de-653dc6203a0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" Jul 09 02:07:52.784459 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:52.784380 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" Jul 09 02:07:52.941051 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:52.941025 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:07:52.941563 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:52.941545 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv"] Jul 09 02:07:52.944741 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:07:52.944720 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b0f85f3_9265_4926_a1de_653dc6203a0e.slice/crio-bd886a5cb0ea91e4b3733f93dfe17975dfd8929a35ae5d109237db8bca4ffe39 WatchSource:0}: Error finding container bd886a5cb0ea91e4b3733f93dfe17975dfd8929a35ae5d109237db8bca4ffe39: Status 404 returned error can't find the container with id bd886a5cb0ea91e4b3733f93dfe17975dfd8929a35ae5d109237db8bca4ffe39 Jul 09 02:07:53.490302 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:53.490262 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:07:53.922833 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:53.922803 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" event={"ID":"6b0f85f3-9265-4926-a1de-653dc6203a0e","Type":"ContainerStarted","Data":"bd886a5cb0ea91e4b3733f93dfe17975dfd8929a35ae5d109237db8bca4ffe39"} Jul 09 02:07:56.679784 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:56.679755 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-x5tmh"] Jul 09 02:07:56.683005 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:56.682986 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x5tmh" Jul 09 02:07:56.686597 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:56.686578 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-5p297\"" Jul 09 02:07:56.695405 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:56.695381 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-x5tmh"] Jul 09 02:07:56.784012 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:56.783981 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzhzr\" (UniqueName: \"kubernetes.io/projected/eb5a1a9f-9cd5-4ebf-a070-ed56eec53f18-kube-api-access-kzhzr\") pod \"authorino-operator-75cc8d5cb7-x5tmh\" (UID: \"eb5a1a9f-9cd5-4ebf-a070-ed56eec53f18\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-x5tmh" Jul 09 02:07:56.884776 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:56.884746 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzhzr\" (UniqueName: \"kubernetes.io/projected/eb5a1a9f-9cd5-4ebf-a070-ed56eec53f18-kube-api-access-kzhzr\") pod \"authorino-operator-75cc8d5cb7-x5tmh\" (UID: \"eb5a1a9f-9cd5-4ebf-a070-ed56eec53f18\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-x5tmh" Jul 09 02:07:56.897262 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:56.897230 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzhzr\" (UniqueName: \"kubernetes.io/projected/eb5a1a9f-9cd5-4ebf-a070-ed56eec53f18-kube-api-access-kzhzr\") pod \"authorino-operator-75cc8d5cb7-x5tmh\" (UID: \"eb5a1a9f-9cd5-4ebf-a070-ed56eec53f18\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-x5tmh" Jul 09 02:07:56.993891 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:56.993802 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x5tmh" Jul 09 02:07:57.134089 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:57.134068 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-x5tmh"] Jul 09 02:07:57.136224 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:07:57.136201 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb5a1a9f_9cd5_4ebf_a070_ed56eec53f18.slice/crio-04b9828b1ca7eb4b2203bde95b844bd2a1d78b063923c98fc106eb2d6dcd8bf6 WatchSource:0}: Error finding container 04b9828b1ca7eb4b2203bde95b844bd2a1d78b063923c98fc106eb2d6dcd8bf6: Status 404 returned error can't find the container with id 04b9828b1ca7eb4b2203bde95b844bd2a1d78b063923c98fc106eb2d6dcd8bf6 Jul 09 02:07:57.355425 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:57.355323 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:07:57.477763 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:57.477728 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:07:57.945329 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:57.945291 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x5tmh" event={"ID":"eb5a1a9f-9cd5-4ebf-a070-ed56eec53f18","Type":"ContainerStarted","Data":"04b9828b1ca7eb4b2203bde95b844bd2a1d78b063923c98fc106eb2d6dcd8bf6"} Jul 09 02:07:58.591734 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:07:58.591695 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:00.911740 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:00.911713 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lngv4" Jul 09 02:08:00.962514 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:00.962484 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" event={"ID":"6b0f85f3-9265-4926-a1de-653dc6203a0e","Type":"ContainerStarted","Data":"0443633b546bce56c80573aba1828ba9f5aa6f05d9116f524fa5f37f272238ba"} Jul 09 02:08:00.962740 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:00.962550 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" Jul 09 02:08:00.964401 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:00.964370 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x5tmh" event={"ID":"eb5a1a9f-9cd5-4ebf-a070-ed56eec53f18","Type":"ContainerStarted","Data":"5753047d93fca04b24d288eb8938f30b3eabc7e1c8fc84f1aa4a86a77c7d4219"} Jul 09 02:08:00.964507 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:00.964498 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x5tmh" Jul 09 02:08:00.983585 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:00.983545 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" podStartSLOduration=1.361630412 podStartE2EDuration="8.983531666s" totalImagesPullingTime="7.621901254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:07:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:08:00.9811912 +0000 UTC m=+501.679245013" watchObservedRunningTime="2026-07-09 02:08:00.983531666 +0000 UTC m=+501.681585478" Jul 09 02:08:01.000529 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:01.000489 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x5tmh" podStartSLOduration=1.528674359 podStartE2EDuration="5.0004771s" totalImagesPullingTime="3.471802741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:07:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:08:01.000069398 +0000 UTC m=+501.698123236" watchObservedRunningTime="2026-07-09 02:08:01.0004771 +0000 UTC m=+501.698530932" Jul 09 02:08:01.402300 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:01.402269 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:01.474402 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:01.474367 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb"] Jul 09 02:08:01.477629 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:01.477613 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb" Jul 09 02:08:01.482182 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:01.482160 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-j59hj\"" Jul 09 02:08:01.500310 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:01.500289 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb"] Jul 09 02:08:01.623342 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:01.623312 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6qtx\" (UniqueName: \"kubernetes.io/projected/19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b-kube-api-access-z6qtx\") pod \"limitador-operator-controller-manager-859d7b78bd-kr9pb\" (UID: \"19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb" Jul 09 02:08:01.723955 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:01.723811 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6qtx\" (UniqueName: \"kubernetes.io/projected/19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b-kube-api-access-z6qtx\") pod \"limitador-operator-controller-manager-859d7b78bd-kr9pb\" (UID: \"19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb" Jul 09 02:08:01.733026 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:01.732996 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6qtx\" (UniqueName: \"kubernetes.io/projected/19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b-kube-api-access-z6qtx\") pod \"limitador-operator-controller-manager-859d7b78bd-kr9pb\" (UID: \"19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb" Jul 09 02:08:01.787567 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:01.787530 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb" Jul 09 02:08:01.944413 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:01.944379 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb"] Jul 09 02:08:01.945353 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:08:01.945330 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19372d1d_4d63_4a1d_a0c6_2dc3dea6b75b.slice/crio-d6be7d1ac30c002a8d539c00af68a4947a68d822bdf5c47986f4b8f5a96ce3a3 WatchSource:0}: Error finding container d6be7d1ac30c002a8d539c00af68a4947a68d822bdf5c47986f4b8f5a96ce3a3: Status 404 returned error can't find the container with id d6be7d1ac30c002a8d539c00af68a4947a68d822bdf5c47986f4b8f5a96ce3a3 Jul 09 02:08:01.970549 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:01.970513 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb" event={"ID":"19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b","Type":"ContainerStarted","Data":"d6be7d1ac30c002a8d539c00af68a4947a68d822bdf5c47986f4b8f5a96ce3a3"} Jul 09 02:08:03.374718 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:03.374691 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:03.475016 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:03.474988 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:03.915933 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:03.915899 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:03.982850 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:03.982775 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb" event={"ID":"19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b","Type":"ContainerStarted","Data":"467b419184665e5ae4af7373a99a653044ba03efdad9d5e84466cb1b28b3719d"} Jul 09 02:08:03.983001 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:03.982920 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb" Jul 09 02:08:04.002508 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:04.002459 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb" podStartSLOduration=1.228808345 podStartE2EDuration="3.002443988s" totalImagesPullingTime="1.773635643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:08:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:08:04.00216375 +0000 UTC m=+504.700217586" watchObservedRunningTime="2026-07-09 02:08:04.002443988 +0000 UTC m=+504.700497795" Jul 09 02:08:04.007001 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:04.006962 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:07.706047 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:07.705993 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:07.768168 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:07.768136 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:11.973265 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:11.973235 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x5tmh" Jul 09 02:08:11.973683 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:11.973302 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" Jul 09 02:08:13.800624 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.800591 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv"] Jul 09 02:08:13.801044 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.800772 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" podUID="6b0f85f3-9265-4926-a1de-653dc6203a0e" containerName="manager" containerID="cri-o://0443633b546bce56c80573aba1828ba9f5aa6f05d9116f524fa5f37f272238ba" gracePeriod=2 Jul 09 02:08:13.813514 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.813490 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv"] Jul 09 02:08:13.828729 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:08:13.828697 2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"6b0f85f3-9265-4926-a1de-653dc6203a0e\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-mc7qv\": pods \"kuadrant-operator-controller-manager-8684c9ff49-mc7qv\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" Jul 09 02:08:13.832742 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.832721 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb"] Jul 09 02:08:13.832987 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.832932 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb" podUID="19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b" containerName="manager" containerID="cri-o://467b419184665e5ae4af7373a99a653044ba03efdad9d5e84466cb1b28b3719d" gracePeriod=2 Jul 09 02:08:13.834821 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.834798 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb" Jul 09 02:08:13.835964 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:08:13.835917 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mc7qv\" is forbidden: User \"system:node:ip-10-0-130-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-184.ec2.internal' and this object" podUID="6b0f85f3-9265-4926-a1de-653dc6203a0e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" Jul 09 02:08:13.838131 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.838104 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb"] Jul 09 02:08:13.838625 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:08:13.838601 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kr9pb\" is forbidden: User \"system:node:ip-10-0-130-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-184.ec2.internal' and this object" podUID="19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb" Jul 09 02:08:13.841438 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:08:13.841405 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mc7qv\" is forbidden: User \"system:node:ip-10-0-130-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-184.ec2.internal' and this object" podUID="6b0f85f3-9265-4926-a1de-653dc6203a0e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" Jul 09 02:08:13.844361 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:08:13.844335 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kr9pb\" is forbidden: User \"system:node:ip-10-0-130-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-184.ec2.internal' and this object" podUID="19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb" Jul 09 02:08:13.848997 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.848977 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l"] Jul 09 02:08:13.849261 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.849248 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b" containerName="manager" Jul 09 02:08:13.849261 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.849261 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b" containerName="manager" Jul 09 02:08:13.849385 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.849271 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b0f85f3-9265-4926-a1de-653dc6203a0e" containerName="manager" Jul 09 02:08:13.849385 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.849277 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b0f85f3-9265-4926-a1de-653dc6203a0e" containerName="manager" Jul 09 02:08:13.849385 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.849344 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b0f85f3-9265-4926-a1de-653dc6203a0e" containerName="manager" Jul 09 02:08:13.849385 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.849351 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b" containerName="manager" Jul 09 02:08:13.852130 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.852112 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l" Jul 09 02:08:13.855034 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:08:13.855010 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mc7qv\" is forbidden: User \"system:node:ip-10-0-130-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-184.ec2.internal' and this object" podUID="6b0f85f3-9265-4926-a1de-653dc6203a0e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" Jul 09 02:08:13.857743 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:08:13.857712 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kr9pb\" is forbidden: User \"system:node:ip-10-0-130-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-184.ec2.internal' and this object" podUID="19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb" Jul 09 02:08:13.874779 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.874757 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l"] Jul 09 02:08:13.891549 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.891527 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q8zh5"] Jul 09 02:08:13.894662 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.894645 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q8zh5" Jul 09 02:08:13.915639 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:13.915616 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q8zh5"] Jul 09 02:08:14.019410 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.019385 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jztr\" (UniqueName: \"kubernetes.io/projected/8f5752c8-a347-415c-9cb4-5224c81bd875-kube-api-access-7jztr\") pod \"limitador-operator-controller-manager-859d7b78bd-q8zh5\" (UID: \"8f5752c8-a347-415c-9cb4-5224c81bd875\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q8zh5" Jul 09 02:08:14.019798 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.019618 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlbj7\" (UniqueName: \"kubernetes.io/projected/e66648cd-b669-4f14-9441-4346ec86c538-kube-api-access-vlbj7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n685l\" (UID: \"e66648cd-b669-4f14-9441-4346ec86c538\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l" Jul 09 02:08:14.020102 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.019937 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e66648cd-b669-4f14-9441-4346ec86c538-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n685l\" (UID: \"e66648cd-b669-4f14-9441-4346ec86c538\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l" Jul 09 02:08:14.030947 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.030921 2582 generic.go:356] "Generic (PLEG): container finished" podID="19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b" containerID="467b419184665e5ae4af7373a99a653044ba03efdad9d5e84466cb1b28b3719d" exitCode=0 Jul 09 02:08:14.032742 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.032716 2582 generic.go:356] "Generic (PLEG): container finished" podID="6b0f85f3-9265-4926-a1de-653dc6203a0e" containerID="0443633b546bce56c80573aba1828ba9f5aa6f05d9116f524fa5f37f272238ba" exitCode=0 Jul 09 02:08:14.078059 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.078039 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" Jul 09 02:08:14.081081 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.081063 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb" Jul 09 02:08:14.120647 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.120618 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlbj7\" (UniqueName: \"kubernetes.io/projected/e66648cd-b669-4f14-9441-4346ec86c538-kube-api-access-vlbj7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n685l\" (UID: \"e66648cd-b669-4f14-9441-4346ec86c538\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l" Jul 09 02:08:14.120781 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.120698 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e66648cd-b669-4f14-9441-4346ec86c538-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n685l\" (UID: \"e66648cd-b669-4f14-9441-4346ec86c538\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l" Jul 09 02:08:14.120781 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.120727 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jztr\" (UniqueName: \"kubernetes.io/projected/8f5752c8-a347-415c-9cb4-5224c81bd875-kube-api-access-7jztr\") pod \"limitador-operator-controller-manager-859d7b78bd-q8zh5\" (UID: \"8f5752c8-a347-415c-9cb4-5224c81bd875\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q8zh5" Jul 09 02:08:14.121185 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.121163 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e66648cd-b669-4f14-9441-4346ec86c538-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n685l\" (UID: \"e66648cd-b669-4f14-9441-4346ec86c538\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l" Jul 09 02:08:14.122582 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.122551 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh"] Jul 09 02:08:14.125737 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.125721 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh" Jul 09 02:08:14.135499 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.135478 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlbj7\" (UniqueName: \"kubernetes.io/projected/e66648cd-b669-4f14-9441-4346ec86c538-kube-api-access-vlbj7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n685l\" (UID: \"e66648cd-b669-4f14-9441-4346ec86c538\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l" Jul 09 02:08:14.135574 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.135550 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jztr\" (UniqueName: \"kubernetes.io/projected/8f5752c8-a347-415c-9cb4-5224c81bd875-kube-api-access-7jztr\") pod \"limitador-operator-controller-manager-859d7b78bd-q8zh5\" (UID: \"8f5752c8-a347-415c-9cb4-5224c81bd875\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q8zh5" Jul 09 02:08:14.142406 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.142388 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh"] Jul 09 02:08:14.221291 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.221269 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvbpv\" (UniqueName: \"kubernetes.io/projected/6b0f85f3-9265-4926-a1de-653dc6203a0e-kube-api-access-cvbpv\") pod \"6b0f85f3-9265-4926-a1de-653dc6203a0e\" (UID: \"6b0f85f3-9265-4926-a1de-653dc6203a0e\") " Jul 09 02:08:14.221391 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.221301 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b0f85f3-9265-4926-a1de-653dc6203a0e-extensions-socket-volume\") pod \"6b0f85f3-9265-4926-a1de-653dc6203a0e\" (UID: \"6b0f85f3-9265-4926-a1de-653dc6203a0e\") " Jul 09 02:08:14.221391 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.221321 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6qtx\" (UniqueName: \"kubernetes.io/projected/19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b-kube-api-access-z6qtx\") pod \"19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b\" (UID: \"19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b\") " Jul 09 02:08:14.221466 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.221439 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35bd079f-3913-46ac-8ac3-d71cf740a453-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qzrlh\" (UID: \"35bd079f-3913-46ac-8ac3-d71cf740a453\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh" Jul 09 02:08:14.221515 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.221495 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tngn7\" (UniqueName: \"kubernetes.io/projected/35bd079f-3913-46ac-8ac3-d71cf740a453-kube-api-access-tngn7\") pod \"kuadrant-operator-controller-manager-86878c769f-qzrlh\" (UID: \"35bd079f-3913-46ac-8ac3-d71cf740a453\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh" Jul 09 02:08:14.221830 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.221810 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b0f85f3-9265-4926-a1de-653dc6203a0e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6b0f85f3-9265-4926-a1de-653dc6203a0e" (UID: "6b0f85f3-9265-4926-a1de-653dc6203a0e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:08:14.223255 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.223236 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b0f85f3-9265-4926-a1de-653dc6203a0e-kube-api-access-cvbpv" (OuterVolumeSpecName: "kube-api-access-cvbpv") pod "6b0f85f3-9265-4926-a1de-653dc6203a0e" (UID: "6b0f85f3-9265-4926-a1de-653dc6203a0e"). InnerVolumeSpecName "kube-api-access-cvbpv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:08:14.223313 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.223241 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l" Jul 09 02:08:14.223313 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.223264 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b-kube-api-access-z6qtx" (OuterVolumeSpecName: "kube-api-access-z6qtx") pod "19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b" (UID: "19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b"). InnerVolumeSpecName "kube-api-access-z6qtx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:08:14.229298 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.229280 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q8zh5" Jul 09 02:08:14.323374 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.323338 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35bd079f-3913-46ac-8ac3-d71cf740a453-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qzrlh\" (UID: \"35bd079f-3913-46ac-8ac3-d71cf740a453\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh" Jul 09 02:08:14.323520 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.323431 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tngn7\" (UniqueName: \"kubernetes.io/projected/35bd079f-3913-46ac-8ac3-d71cf740a453-kube-api-access-tngn7\") pod \"kuadrant-operator-controller-manager-86878c769f-qzrlh\" (UID: \"35bd079f-3913-46ac-8ac3-d71cf740a453\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh" Jul 09 02:08:14.323520 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.323488 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvbpv\" (UniqueName: \"kubernetes.io/projected/6b0f85f3-9265-4926-a1de-653dc6203a0e-kube-api-access-cvbpv\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:08:14.323520 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.323503 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b0f85f3-9265-4926-a1de-653dc6203a0e-extensions-socket-volume\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:08:14.323520 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.323517 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6qtx\" (UniqueName: \"kubernetes.io/projected/19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b-kube-api-access-z6qtx\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:08:14.323813 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.323790 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35bd079f-3913-46ac-8ac3-d71cf740a453-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qzrlh\" (UID: \"35bd079f-3913-46ac-8ac3-d71cf740a453\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh" Jul 09 02:08:14.344974 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.344947 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tngn7\" (UniqueName: \"kubernetes.io/projected/35bd079f-3913-46ac-8ac3-d71cf740a453-kube-api-access-tngn7\") pod \"kuadrant-operator-controller-manager-86878c769f-qzrlh\" (UID: \"35bd079f-3913-46ac-8ac3-d71cf740a453\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh" Jul 09 02:08:14.373590 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.373564 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l"] Jul 09 02:08:14.375352 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:08:14.375329 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode66648cd_b669_4f14_9441_4346ec86c538.slice/crio-0593061a6fd831848da1c0a8792aa4a47e8866031eaed1dbfb72b9365c995258 WatchSource:0}: Error finding container 0593061a6fd831848da1c0a8792aa4a47e8866031eaed1dbfb72b9365c995258: Status 404 returned error can't find the container with id 0593061a6fd831848da1c0a8792aa4a47e8866031eaed1dbfb72b9365c995258 Jul 09 02:08:14.385994 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.385973 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q8zh5"] Jul 09 02:08:14.388832 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:08:14.388812 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f5752c8_a347_415c_9cb4_5224c81bd875.slice/crio-b16f273f7b883cc3ba555dc900cabe461d856e6685226f2c492f9c00404a50c6 WatchSource:0}: Error finding container b16f273f7b883cc3ba555dc900cabe461d856e6685226f2c492f9c00404a50c6: Status 404 returned error can't find the container with id b16f273f7b883cc3ba555dc900cabe461d856e6685226f2c492f9c00404a50c6 Jul 09 02:08:14.435250 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.435229 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh" Jul 09 02:08:14.572310 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:14.572263 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh"] Jul 09 02:08:14.574529 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:08:14.574504 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35bd079f_3913_46ac_8ac3_d71cf740a453.slice/crio-09061796292dd2ad4335e02edbcac56fcb03c3d1d25a56301f6b34e000a4cebf WatchSource:0}: Error finding container 09061796292dd2ad4335e02edbcac56fcb03c3d1d25a56301f6b34e000a4cebf: Status 404 returned error can't find the container with id 09061796292dd2ad4335e02edbcac56fcb03c3d1d25a56301f6b34e000a4cebf Jul 09 02:08:15.041486 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:15.041455 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l" event={"ID":"e66648cd-b669-4f14-9441-4346ec86c538","Type":"ContainerStarted","Data":"724ac75becbc5ad95dbcd4aa65174028f6f30cdb231683b2f8f9366d1eeed335"} Jul 09 02:08:15.041918 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:15.041509 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l" Jul 09 02:08:15.041918 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:15.041532 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l" event={"ID":"e66648cd-b669-4f14-9441-4346ec86c538","Type":"ContainerStarted","Data":"0593061a6fd831848da1c0a8792aa4a47e8866031eaed1dbfb72b9365c995258"} Jul 09 02:08:15.043087 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:15.043066 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh" event={"ID":"35bd079f-3913-46ac-8ac3-d71cf740a453","Type":"ContainerStarted","Data":"192c31167ded0f69047f3339f11a6fd11a5288396149ec53f33d45f3fa2f51b6"} Jul 09 02:08:15.043198 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:15.043103 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh" event={"ID":"35bd079f-3913-46ac-8ac3-d71cf740a453","Type":"ContainerStarted","Data":"09061796292dd2ad4335e02edbcac56fcb03c3d1d25a56301f6b34e000a4cebf"} Jul 09 02:08:15.043198 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:15.043131 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh" Jul 09 02:08:15.044679 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:15.044659 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q8zh5" event={"ID":"8f5752c8-a347-415c-9cb4-5224c81bd875","Type":"ContainerStarted","Data":"3f4d682ad2057795ae368d8619e8248e38ef535e8129fd22194b4e76b2d3c483"} Jul 09 02:08:15.044773 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:15.044691 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q8zh5" event={"ID":"8f5752c8-a347-415c-9cb4-5224c81bd875","Type":"ContainerStarted","Data":"b16f273f7b883cc3ba555dc900cabe461d856e6685226f2c492f9c00404a50c6"} Jul 09 02:08:15.044773 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:15.044733 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q8zh5" Jul 09 02:08:15.045817 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:15.045803 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb" Jul 09 02:08:15.045924 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:15.045829 2582 scope.go:117] "RemoveContainer" containerID="467b419184665e5ae4af7373a99a653044ba03efdad9d5e84466cb1b28b3719d" Jul 09 02:08:15.047031 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:15.046983 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mc7qv" Jul 09 02:08:15.054942 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:15.054928 2582 scope.go:117] "RemoveContainer" containerID="0443633b546bce56c80573aba1828ba9f5aa6f05d9116f524fa5f37f272238ba" Jul 09 02:08:15.070456 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:15.070414 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l" podStartSLOduration=2.07039724 podStartE2EDuration="2.07039724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:08:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:08:15.069837546 +0000 UTC m=+515.767891371" watchObservedRunningTime="2026-07-09 02:08:15.07039724 +0000 UTC m=+515.768451055" Jul 09 02:08:15.072513 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:08:15.072489 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kr9pb\" is forbidden: User \"system:node:ip-10-0-130-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-184.ec2.internal' and this object" podUID="19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kr9pb" Jul 09 02:08:15.101303 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:15.101235 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q8zh5" podStartSLOduration=2.101222467 podStartE2EDuration="2.101222467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:08:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:08:15.100244911 +0000 UTC m=+515.798298724" watchObservedRunningTime="2026-07-09 02:08:15.101222467 +0000 UTC m=+515.799276280" Jul 09 02:08:15.125953 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:15.125909 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh" podStartSLOduration=1.125896677 podStartE2EDuration="1.125896677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:08:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:08:15.125017628 +0000 UTC m=+515.823071440" watchObservedRunningTime="2026-07-09 02:08:15.125896677 +0000 UTC m=+515.823950490" Jul 09 02:08:15.866116 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:15.866080 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b" path="/var/lib/kubelet/pods/19372d1d-4d63-4a1d-a0c6-2dc3dea6b75b/volumes" Jul 09 02:08:15.866427 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:15.866414 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b0f85f3-9265-4926-a1de-653dc6203a0e" path="/var/lib/kubelet/pods/6b0f85f3-9265-4926-a1de-653dc6203a0e/volumes" Jul 09 02:08:16.727383 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:16.727354 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:16.828291 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:16.828259 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:16.927399 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:16.927371 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:19.721377 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:19.721345 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:19.826421 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:19.826393 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:22.322578 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:22.322543 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:22.424117 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:22.424082 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:26.056565 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:26.056534 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q8zh5" Jul 09 02:08:26.056955 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:26.056609 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh" Jul 09 02:08:26.056955 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:26.056756 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l" Jul 09 02:08:26.146585 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:26.146558 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l"] Jul 09 02:08:26.146763 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:26.146722 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l" podUID="e66648cd-b669-4f14-9441-4346ec86c538" containerName="manager" containerID="cri-o://724ac75becbc5ad95dbcd4aa65174028f6f30cdb231683b2f8f9366d1eeed335" gracePeriod=10 Jul 09 02:08:26.152633 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:26.152577 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:26.241943 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:26.241912 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:26.382903 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:26.382858 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l" Jul 09 02:08:26.421859 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:26.421829 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e66648cd-b669-4f14-9441-4346ec86c538-extensions-socket-volume\") pod \"e66648cd-b669-4f14-9441-4346ec86c538\" (UID: \"e66648cd-b669-4f14-9441-4346ec86c538\") " Jul 09 02:08:26.422024 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:26.421889 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlbj7\" (UniqueName: \"kubernetes.io/projected/e66648cd-b669-4f14-9441-4346ec86c538-kube-api-access-vlbj7\") pod \"e66648cd-b669-4f14-9441-4346ec86c538\" (UID: \"e66648cd-b669-4f14-9441-4346ec86c538\") " Jul 09 02:08:26.422193 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:26.422170 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e66648cd-b669-4f14-9441-4346ec86c538-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e66648cd-b669-4f14-9441-4346ec86c538" (UID: "e66648cd-b669-4f14-9441-4346ec86c538"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:08:26.423827 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:26.423791 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e66648cd-b669-4f14-9441-4346ec86c538-kube-api-access-vlbj7" (OuterVolumeSpecName: "kube-api-access-vlbj7") pod "e66648cd-b669-4f14-9441-4346ec86c538" (UID: "e66648cd-b669-4f14-9441-4346ec86c538"). InnerVolumeSpecName "kube-api-access-vlbj7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:08:26.523241 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:26.523212 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e66648cd-b669-4f14-9441-4346ec86c538-extensions-socket-volume\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:08:26.523241 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:26.523236 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlbj7\" (UniqueName: \"kubernetes.io/projected/e66648cd-b669-4f14-9441-4346ec86c538-kube-api-access-vlbj7\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:08:27.116709 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:27.116676 2582 generic.go:356] "Generic (PLEG): container finished" podID="e66648cd-b669-4f14-9441-4346ec86c538" containerID="724ac75becbc5ad95dbcd4aa65174028f6f30cdb231683b2f8f9366d1eeed335" exitCode=0 Jul 09 02:08:27.117174 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:27.116717 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l" Jul 09 02:08:27.117174 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:27.116738 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l" event={"ID":"e66648cd-b669-4f14-9441-4346ec86c538","Type":"ContainerDied","Data":"724ac75becbc5ad95dbcd4aa65174028f6f30cdb231683b2f8f9366d1eeed335"} Jul 09 02:08:27.117174 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:27.116779 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l" event={"ID":"e66648cd-b669-4f14-9441-4346ec86c538","Type":"ContainerDied","Data":"0593061a6fd831848da1c0a8792aa4a47e8866031eaed1dbfb72b9365c995258"} Jul 09 02:08:27.117174 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:27.116801 2582 scope.go:117] "RemoveContainer" containerID="724ac75becbc5ad95dbcd4aa65174028f6f30cdb231683b2f8f9366d1eeed335" Jul 09 02:08:27.126274 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:27.126252 2582 scope.go:117] "RemoveContainer" containerID="724ac75becbc5ad95dbcd4aa65174028f6f30cdb231683b2f8f9366d1eeed335" Jul 09 02:08:27.126536 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:08:27.126516 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"724ac75becbc5ad95dbcd4aa65174028f6f30cdb231683b2f8f9366d1eeed335\": container with ID starting with 724ac75becbc5ad95dbcd4aa65174028f6f30cdb231683b2f8f9366d1eeed335 not found: ID does not exist" containerID="724ac75becbc5ad95dbcd4aa65174028f6f30cdb231683b2f8f9366d1eeed335" Jul 09 02:08:27.126588 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:27.126548 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"724ac75becbc5ad95dbcd4aa65174028f6f30cdb231683b2f8f9366d1eeed335"} err="failed to get container status \"724ac75becbc5ad95dbcd4aa65174028f6f30cdb231683b2f8f9366d1eeed335\": rpc error: code = NotFound desc = could not find container \"724ac75becbc5ad95dbcd4aa65174028f6f30cdb231683b2f8f9366d1eeed335\": container with ID starting with 724ac75becbc5ad95dbcd4aa65174028f6f30cdb231683b2f8f9366d1eeed335 not found: ID does not exist" Jul 09 02:08:27.143694 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:27.143668 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l"] Jul 09 02:08:27.155881 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:27.152377 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n685l"] Jul 09 02:08:27.865266 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:27.865238 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e66648cd-b669-4f14-9441-4346ec86c538" path="/var/lib/kubelet/pods/e66648cd-b669-4f14-9441-4346ec86c538/volumes" Jul 09 02:08:34.935103 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:34.935015 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:34.994955 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:34.994924 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:35.143347 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:35.143317 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:35.234882 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:35.234798 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:35.380349 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:35.380318 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:37.737222 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:37.737196 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:37.837612 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:37.837582 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:42.572689 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:42.572659 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:42.756351 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:42.756318 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:45.008787 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.008758 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:45.121522 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.121485 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:45.222153 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.222098 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:45.333615 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.333532 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:08:45.333979 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.333962 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e66648cd-b669-4f14-9441-4346ec86c538" containerName="manager" Jul 09 02:08:45.334077 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.333982 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e66648cd-b669-4f14-9441-4346ec86c538" containerName="manager" Jul 09 02:08:45.334131 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.334091 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e66648cd-b669-4f14-9441-4346ec86c538" containerName="manager" Jul 09 02:08:45.338446 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.338423 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-r7rtl" Jul 09 02:08:45.341589 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.341559 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 09 02:08:45.341589 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.341587 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xqpgc\"" Jul 09 02:08:45.347373 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.347350 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:08:45.376262 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.376237 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:08:45.459889 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.459852 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/300bc103-5b60-4d66-9418-885fa3e554bb-config-file\") pod \"limitador-limitador-54d6d988d5-r7rtl\" (UID: \"300bc103-5b60-4d66-9418-885fa3e554bb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-r7rtl" Jul 09 02:08:45.460058 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.459986 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kbjz\" (UniqueName: \"kubernetes.io/projected/300bc103-5b60-4d66-9418-885fa3e554bb-kube-api-access-7kbjz\") pod \"limitador-limitador-54d6d988d5-r7rtl\" (UID: \"300bc103-5b60-4d66-9418-885fa3e554bb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-r7rtl" Jul 09 02:08:45.561163 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.561134 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/300bc103-5b60-4d66-9418-885fa3e554bb-config-file\") pod \"limitador-limitador-54d6d988d5-r7rtl\" (UID: \"300bc103-5b60-4d66-9418-885fa3e554bb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-r7rtl" Jul 09 02:08:45.561331 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.561207 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kbjz\" (UniqueName: \"kubernetes.io/projected/300bc103-5b60-4d66-9418-885fa3e554bb-kube-api-access-7kbjz\") pod \"limitador-limitador-54d6d988d5-r7rtl\" (UID: \"300bc103-5b60-4d66-9418-885fa3e554bb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-r7rtl" Jul 09 02:08:45.561760 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.561738 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/300bc103-5b60-4d66-9418-885fa3e554bb-config-file\") pod \"limitador-limitador-54d6d988d5-r7rtl\" (UID: \"300bc103-5b60-4d66-9418-885fa3e554bb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-r7rtl" Jul 09 02:08:45.570342 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.570315 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kbjz\" (UniqueName: \"kubernetes.io/projected/300bc103-5b60-4d66-9418-885fa3e554bb-kube-api-access-7kbjz\") pod \"limitador-limitador-54d6d988d5-r7rtl\" (UID: \"300bc103-5b60-4d66-9418-885fa3e554bb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-r7rtl" Jul 09 02:08:45.649983 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.649958 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-r7rtl" Jul 09 02:08:45.776419 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.776395 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:08:45.778264 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:08:45.778229 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod300bc103_5b60_4d66_9418_885fa3e554bb.slice/crio-fa10cf0c437e690f338311b4b5a2dad77c6f29a1b8255dc7340b8af527a27615 WatchSource:0}: Error finding container fa10cf0c437e690f338311b4b5a2dad77c6f29a1b8255dc7340b8af527a27615: Status 404 returned error can't find the container with id fa10cf0c437e690f338311b4b5a2dad77c6f29a1b8255dc7340b8af527a27615 Jul 09 02:08:45.834340 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.834313 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:45.910048 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:45.909979 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:08:46.212268 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:46.212189 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-r7rtl" event={"ID":"300bc103-5b60-4d66-9418-885fa3e554bb","Type":"ContainerStarted","Data":"fa10cf0c437e690f338311b4b5a2dad77c6f29a1b8255dc7340b8af527a27615"} Jul 09 02:08:49.231087 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:49.231053 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-r7rtl" event={"ID":"300bc103-5b60-4d66-9418-885fa3e554bb","Type":"ContainerStarted","Data":"418340a2f0a96ecd3fb043537b3527db3f70f5f780ef51be5555c8a012d5297d"} Jul 09 02:08:49.231443 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:49.231112 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-r7rtl" Jul 09 02:08:49.249648 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:08:49.249598 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-r7rtl" podStartSLOduration=1.733792852 podStartE2EDuration="4.249585536s" totalImagesPullingTime="2.515792684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:08:49.249057066 +0000 UTC m=+549.947110890" watchObservedRunningTime="2026-07-09 02:08:49.249585536 +0000 UTC m=+549.947639348" Jul 09 02:09:00.236810 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:00.236777 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-r7rtl" Jul 09 02:09:06.268898 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:06.268833 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:09:06.365067 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:06.365038 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:09:06.814051 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:06.814022 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:09:20.512787 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:20.512755 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:09:20.648743 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:20.648712 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:09:27.420924 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:27.420890 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:09:27.519430 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:27.519403 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:09:27.606836 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:27.606807 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:09:27.611352 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:27.611326 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-669bbdcddd-f46ph"] Jul 09 02:09:27.622199 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:27.622175 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-669bbdcddd-f46ph" Jul 09 02:09:27.625108 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:27.625082 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-669bbdcddd-f46ph"] Jul 09 02:09:27.625250 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:27.625194 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 02:09:27.625347 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:27.625328 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 02:09:27.625416 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:27.625386 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m5h9z\"" Jul 09 02:09:27.752456 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:27.752379 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:09:27.793540 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:27.793518 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvmlc\" (UniqueName: \"kubernetes.io/projected/2bab31b7-6129-4d28-bffc-95ef653cc648-kube-api-access-fvmlc\") pod \"maas-controller-669bbdcddd-f46ph\" (UID: \"2bab31b7-6129-4d28-bffc-95ef653cc648\") " pod="opendatahub/maas-controller-669bbdcddd-f46ph" Jul 09 02:09:27.793624 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:27.793577 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bab31b7-6129-4d28-bffc-95ef653cc648-webhook-cert\") pod \"maas-controller-669bbdcddd-f46ph\" (UID: \"2bab31b7-6129-4d28-bffc-95ef653cc648\") " pod="opendatahub/maas-controller-669bbdcddd-f46ph" Jul 09 02:09:27.894042 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:27.894014 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bab31b7-6129-4d28-bffc-95ef653cc648-webhook-cert\") pod \"maas-controller-669bbdcddd-f46ph\" (UID: \"2bab31b7-6129-4d28-bffc-95ef653cc648\") " pod="opendatahub/maas-controller-669bbdcddd-f46ph" Jul 09 02:09:27.894169 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:27.894069 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvmlc\" (UniqueName: \"kubernetes.io/projected/2bab31b7-6129-4d28-bffc-95ef653cc648-kube-api-access-fvmlc\") pod \"maas-controller-669bbdcddd-f46ph\" (UID: \"2bab31b7-6129-4d28-bffc-95ef653cc648\") " pod="opendatahub/maas-controller-669bbdcddd-f46ph" Jul 09 02:09:27.896355 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:27.896337 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bab31b7-6129-4d28-bffc-95ef653cc648-webhook-cert\") pod \"maas-controller-669bbdcddd-f46ph\" (UID: \"2bab31b7-6129-4d28-bffc-95ef653cc648\") " pod="opendatahub/maas-controller-669bbdcddd-f46ph" Jul 09 02:09:27.902704 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:27.902683 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvmlc\" (UniqueName: \"kubernetes.io/projected/2bab31b7-6129-4d28-bffc-95ef653cc648-kube-api-access-fvmlc\") pod \"maas-controller-669bbdcddd-f46ph\" (UID: \"2bab31b7-6129-4d28-bffc-95ef653cc648\") " pod="opendatahub/maas-controller-669bbdcddd-f46ph" Jul 09 02:09:27.934418 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:27.934396 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-669bbdcddd-f46ph" Jul 09 02:09:28.061415 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:28.061380 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-669bbdcddd-f46ph"] Jul 09 02:09:28.063789 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:09:28.063761 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bab31b7_6129_4d28_bffc_95ef653cc648.slice/crio-1dfa3685f9e7555962f1be4ff04584afe730b7d3bf5c85fa8fe1b18760658c31 WatchSource:0}: Error finding container 1dfa3685f9e7555962f1be4ff04584afe730b7d3bf5c85fa8fe1b18760658c31: Status 404 returned error can't find the container with id 1dfa3685f9e7555962f1be4ff04584afe730b7d3bf5c85fa8fe1b18760658c31 Jul 09 02:09:28.418779 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:28.418747 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-669bbdcddd-f46ph" event={"ID":"2bab31b7-6129-4d28-bffc-95ef653cc648","Type":"ContainerStarted","Data":"1dfa3685f9e7555962f1be4ff04584afe730b7d3bf5c85fa8fe1b18760658c31"} Jul 09 02:09:30.209311 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:30.209277 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:09:30.298469 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:30.298373 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:09:30.429935 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:30.429901 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-669bbdcddd-f46ph" event={"ID":"2bab31b7-6129-4d28-bffc-95ef653cc648","Type":"ContainerStarted","Data":"ae38028e00cba93eadd3aaaeab82cfd68221ebdd6e858869596c182258a7f71e"} Jul 09 02:09:30.430119 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:30.430008 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-669bbdcddd-f46ph" Jul 09 02:09:30.464768 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:30.464725 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-669bbdcddd-f46ph" podStartSLOduration=1.503006786 podStartE2EDuration="3.464707289s" totalImagesPullingTime="1.961700503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:09:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:09:30.460036578 +0000 UTC m=+591.158090393" watchObservedRunningTime="2026-07-09 02:09:30.464707289 +0000 UTC m=+591.162761102" Jul 09 02:09:30.913636 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:30.913599 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:09:39.719152 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:39.719127 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:09:39.726599 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:39.726582 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:09:41.446349 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:09:41.446317 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-669bbdcddd-f46ph" Jul 09 02:10:04.179568 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:04.179483 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:10:05.166776 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:05.166739 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:10:05.309140 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:05.309104 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:10:15.413421 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:15.413387 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:10:15.506254 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:15.506225 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:10:19.743741 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:19.743705 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:10:20.309316 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:20.309239 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:10:23.753489 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:23.753450 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:10:24.601046 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:24.601011 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:10:25.829320 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:25.829281 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:10:37.345288 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:37.345253 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:10:38.530776 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:38.530734 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:10:42.693691 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.693660 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6"] Jul 09 02:10:42.696555 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.696539 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:42.699989 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.699967 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 02:10:42.700227 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.700210 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tcfth\"" Jul 09 02:10:42.700349 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.700326 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 09 02:10:42.701553 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.701532 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 02:10:42.709143 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.709123 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6"] Jul 09 02:10:42.793552 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.793516 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d246772-b923-4799-8af0-50af7a16d280-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6\" (UID: \"0d246772-b923-4799-8af0-50af7a16d280\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:42.793552 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.793551 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d246772-b923-4799-8af0-50af7a16d280-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6\" (UID: \"0d246772-b923-4799-8af0-50af7a16d280\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:42.793791 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.793588 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d246772-b923-4799-8af0-50af7a16d280-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6\" (UID: \"0d246772-b923-4799-8af0-50af7a16d280\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:42.793791 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.793641 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d246772-b923-4799-8af0-50af7a16d280-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6\" (UID: \"0d246772-b923-4799-8af0-50af7a16d280\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:42.793791 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.793671 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d246772-b923-4799-8af0-50af7a16d280-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6\" (UID: \"0d246772-b923-4799-8af0-50af7a16d280\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:42.793791 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.793735 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnbnt\" (UniqueName: \"kubernetes.io/projected/0d246772-b923-4799-8af0-50af7a16d280-kube-api-access-wnbnt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6\" (UID: \"0d246772-b923-4799-8af0-50af7a16d280\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:42.854114 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.854060 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:10:42.894812 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.894784 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d246772-b923-4799-8af0-50af7a16d280-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6\" (UID: \"0d246772-b923-4799-8af0-50af7a16d280\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:42.894967 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.894816 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d246772-b923-4799-8af0-50af7a16d280-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6\" (UID: \"0d246772-b923-4799-8af0-50af7a16d280\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:42.894967 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.894938 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d246772-b923-4799-8af0-50af7a16d280-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6\" (UID: \"0d246772-b923-4799-8af0-50af7a16d280\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:42.895092 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.894983 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d246772-b923-4799-8af0-50af7a16d280-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6\" (UID: \"0d246772-b923-4799-8af0-50af7a16d280\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:42.895092 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.895044 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d246772-b923-4799-8af0-50af7a16d280-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6\" (UID: \"0d246772-b923-4799-8af0-50af7a16d280\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:42.895216 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.895201 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d246772-b923-4799-8af0-50af7a16d280-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6\" (UID: \"0d246772-b923-4799-8af0-50af7a16d280\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:42.895340 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.895317 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnbnt\" (UniqueName: \"kubernetes.io/projected/0d246772-b923-4799-8af0-50af7a16d280-kube-api-access-wnbnt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6\" (UID: \"0d246772-b923-4799-8af0-50af7a16d280\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:42.895610 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.895584 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d246772-b923-4799-8af0-50af7a16d280-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6\" (UID: \"0d246772-b923-4799-8af0-50af7a16d280\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:42.895688 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.895670 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d246772-b923-4799-8af0-50af7a16d280-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6\" (UID: \"0d246772-b923-4799-8af0-50af7a16d280\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:42.897503 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.897477 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d246772-b923-4799-8af0-50af7a16d280-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6\" (UID: \"0d246772-b923-4799-8af0-50af7a16d280\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:42.897733 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.897717 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d246772-b923-4799-8af0-50af7a16d280-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6\" (UID: \"0d246772-b923-4799-8af0-50af7a16d280\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:42.907559 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:42.907536 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnbnt\" (UniqueName: \"kubernetes.io/projected/0d246772-b923-4799-8af0-50af7a16d280-kube-api-access-wnbnt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6\" (UID: \"0d246772-b923-4799-8af0-50af7a16d280\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:43.007577 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:43.007502 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:43.144833 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:43.144805 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6"] Jul 09 02:10:43.146729 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:10:43.146701 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d246772_b923_4799_8af0_50af7a16d280.slice/crio-0402d63c97197e3b87590bd8e911aa78417e947e647c0a0e70561860221243d5 WatchSource:0}: Error finding container 0402d63c97197e3b87590bd8e911aa78417e947e647c0a0e70561860221243d5: Status 404 returned error can't find the container with id 0402d63c97197e3b87590bd8e911aa78417e947e647c0a0e70561860221243d5 Jul 09 02:10:43.148027 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:43.148010 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 02:10:43.219515 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:43.219490 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:10:43.794122 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:43.794076 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" event={"ID":"0d246772-b923-4799-8af0-50af7a16d280","Type":"ContainerStarted","Data":"0402d63c97197e3b87590bd8e911aa78417e947e647c0a0e70561860221243d5"} Jul 09 02:10:44.800885 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:44.800851 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" event={"ID":"0d246772-b923-4799-8af0-50af7a16d280","Type":"ContainerStarted","Data":"615355aaf1e9b892c16683ec709d6b13712b4919735c775f88e5eff47081f5db"} Jul 09 02:10:44.822064 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:44.822012 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" podStartSLOduration=1.734216878 podStartE2EDuration="2.821995507s" totalImagesPullingTime="1.087778629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:10:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:10:44.820153546 +0000 UTC m=+665.518207359" watchObservedRunningTime="2026-07-09 02:10:44.821995507 +0000 UTC m=+665.520049320" Jul 09 02:10:44.997057 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:44.997024 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:10:51.991463 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:51.991423 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:10:53.007916 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:53.007887 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:53.008299 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:53.007935 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:53.020479 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:53.020450 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:10:53.855252 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:10:53.855226 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6" Jul 09 02:11:03.247649 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:03.247609 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:11:05.397608 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:05.397579 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:11:29.501742 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.501592 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8"] Jul 09 02:11:29.507338 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.507313 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.511737 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.511715 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 09 02:11:29.512120 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.512101 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8"] Jul 09 02:11:29.528091 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.528051 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:11:29.682812 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.682782 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f345d6e-1b9b-440e-a83e-07df6109368b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8\" (UID: \"7f345d6e-1b9b-440e-a83e-07df6109368b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.682812 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.682815 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f345d6e-1b9b-440e-a83e-07df6109368b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8\" (UID: \"7f345d6e-1b9b-440e-a83e-07df6109368b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.683019 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.682846 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f345d6e-1b9b-440e-a83e-07df6109368b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8\" (UID: \"7f345d6e-1b9b-440e-a83e-07df6109368b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.683019 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.682932 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f345d6e-1b9b-440e-a83e-07df6109368b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8\" (UID: \"7f345d6e-1b9b-440e-a83e-07df6109368b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.683019 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.682995 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f345d6e-1b9b-440e-a83e-07df6109368b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8\" (UID: \"7f345d6e-1b9b-440e-a83e-07df6109368b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.683118 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.683025 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckjx8\" (UniqueName: \"kubernetes.io/projected/7f345d6e-1b9b-440e-a83e-07df6109368b-kube-api-access-ckjx8\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8\" (UID: \"7f345d6e-1b9b-440e-a83e-07df6109368b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.783596 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.783517 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f345d6e-1b9b-440e-a83e-07df6109368b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8\" (UID: \"7f345d6e-1b9b-440e-a83e-07df6109368b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.783596 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.783570 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f345d6e-1b9b-440e-a83e-07df6109368b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8\" (UID: \"7f345d6e-1b9b-440e-a83e-07df6109368b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.783804 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.783613 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f345d6e-1b9b-440e-a83e-07df6109368b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8\" (UID: \"7f345d6e-1b9b-440e-a83e-07df6109368b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.783804 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.783638 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckjx8\" (UniqueName: \"kubernetes.io/projected/7f345d6e-1b9b-440e-a83e-07df6109368b-kube-api-access-ckjx8\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8\" (UID: \"7f345d6e-1b9b-440e-a83e-07df6109368b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.783804 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.783701 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f345d6e-1b9b-440e-a83e-07df6109368b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8\" (UID: \"7f345d6e-1b9b-440e-a83e-07df6109368b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.783804 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.783726 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f345d6e-1b9b-440e-a83e-07df6109368b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8\" (UID: \"7f345d6e-1b9b-440e-a83e-07df6109368b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.784092 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.783948 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f345d6e-1b9b-440e-a83e-07df6109368b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8\" (UID: \"7f345d6e-1b9b-440e-a83e-07df6109368b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.784092 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.784057 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f345d6e-1b9b-440e-a83e-07df6109368b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8\" (UID: \"7f345d6e-1b9b-440e-a83e-07df6109368b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.784183 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.784098 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f345d6e-1b9b-440e-a83e-07df6109368b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8\" (UID: \"7f345d6e-1b9b-440e-a83e-07df6109368b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.785942 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.785920 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f345d6e-1b9b-440e-a83e-07df6109368b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8\" (UID: \"7f345d6e-1b9b-440e-a83e-07df6109368b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.786137 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.786122 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f345d6e-1b9b-440e-a83e-07df6109368b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8\" (UID: \"7f345d6e-1b9b-440e-a83e-07df6109368b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.792195 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.792175 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckjx8\" (UniqueName: \"kubernetes.io/projected/7f345d6e-1b9b-440e-a83e-07df6109368b-kube-api-access-ckjx8\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8\" (UID: \"7f345d6e-1b9b-440e-a83e-07df6109368b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.818096 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.818062 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:29.956329 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:29.956306 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8"] Jul 09 02:11:29.958217 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:11:29.958189 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f345d6e_1b9b_440e_a83e_07df6109368b.slice/crio-296cd1e19294eb078e4258e63a5556b63b09eac7dd2d779b31e51111ff77ae80 WatchSource:0}: Error finding container 296cd1e19294eb078e4258e63a5556b63b09eac7dd2d779b31e51111ff77ae80: Status 404 returned error can't find the container with id 296cd1e19294eb078e4258e63a5556b63b09eac7dd2d779b31e51111ff77ae80 Jul 09 02:11:30.013571 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:30.013540 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" event={"ID":"7f345d6e-1b9b-440e-a83e-07df6109368b","Type":"ContainerStarted","Data":"296cd1e19294eb078e4258e63a5556b63b09eac7dd2d779b31e51111ff77ae80"} Jul 09 02:11:31.020501 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:31.020419 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" event={"ID":"7f345d6e-1b9b-440e-a83e-07df6109368b","Type":"ContainerStarted","Data":"f669b86c92c43c633fcb3b83a569c67529adeb3c4ba7310811b92bf97ac7fad4"} Jul 09 02:11:31.040663 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:31.040617 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" podStartSLOduration=1.853351886 podStartE2EDuration="2.040598163s" totalImagesPullingTime="187.246277ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:11:31.038633141 +0000 UTC m=+711.736686966" watchObservedRunningTime="2026-07-09 02:11:31.040598163 +0000 UTC m=+711.738652067" Jul 09 02:11:32.292276 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:32.292247 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:11:39.818171 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:39.818137 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:39.818557 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:39.818184 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:39.831234 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:39.831212 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:11:40.079310 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:11:40.079229 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8" Jul 09 02:12:20.305465 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:12:20.305367 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:12:20.420299 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:12:20.420259 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:12:21.463051 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:12:21.463019 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:12:21.563553 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:12:21.563513 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:12:49.193039 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:12:49.193006 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:12:50.303541 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:12:50.303494 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:12:50.403367 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:12:50.403335 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:12:58.484941 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:12:58.484907 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:13:09.905667 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:09.905593 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:13:18.286864 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:18.286829 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:13:29.186611 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:29.186571 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:13:31.500681 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:31.500640 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:13:31.592928 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:31.592900 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-669bbdcddd-f46ph"] Jul 09 02:13:31.608486 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:31.608436 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-669bbdcddd-f46ph" podUID="2bab31b7-6129-4d28-bffc-95ef653cc648" containerName="manager" containerID="cri-o://ae38028e00cba93eadd3aaaeab82cfd68221ebdd6e858869596c182258a7f71e" gracePeriod=30 Jul 09 02:13:31.851768 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:31.851747 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-669bbdcddd-f46ph" Jul 09 02:13:31.881654 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:31.881633 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bab31b7-6129-4d28-bffc-95ef653cc648-webhook-cert\") pod \"2bab31b7-6129-4d28-bffc-95ef653cc648\" (UID: \"2bab31b7-6129-4d28-bffc-95ef653cc648\") " Jul 09 02:13:31.881778 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:31.881667 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvmlc\" (UniqueName: \"kubernetes.io/projected/2bab31b7-6129-4d28-bffc-95ef653cc648-kube-api-access-fvmlc\") pod \"2bab31b7-6129-4d28-bffc-95ef653cc648\" (UID: \"2bab31b7-6129-4d28-bffc-95ef653cc648\") " Jul 09 02:13:31.883780 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:31.883749 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bab31b7-6129-4d28-bffc-95ef653cc648-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "2bab31b7-6129-4d28-bffc-95ef653cc648" (UID: "2bab31b7-6129-4d28-bffc-95ef653cc648"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 02:13:31.883914 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:31.883893 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bab31b7-6129-4d28-bffc-95ef653cc648-kube-api-access-fvmlc" (OuterVolumeSpecName: "kube-api-access-fvmlc") pod "2bab31b7-6129-4d28-bffc-95ef653cc648" (UID: "2bab31b7-6129-4d28-bffc-95ef653cc648"). InnerVolumeSpecName "kube-api-access-fvmlc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:13:31.982332 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:31.982302 2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bab31b7-6129-4d28-bffc-95ef653cc648-webhook-cert\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:13:31.982332 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:31.982327 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvmlc\" (UniqueName: \"kubernetes.io/projected/2bab31b7-6129-4d28-bffc-95ef653cc648-kube-api-access-fvmlc\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:13:32.614359 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:32.614327 2582 generic.go:356] "Generic (PLEG): container finished" podID="2bab31b7-6129-4d28-bffc-95ef653cc648" containerID="ae38028e00cba93eadd3aaaeab82cfd68221ebdd6e858869596c182258a7f71e" exitCode=0 Jul 09 02:13:32.614791 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:32.614367 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-669bbdcddd-f46ph" Jul 09 02:13:32.614791 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:32.614417 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-669bbdcddd-f46ph" event={"ID":"2bab31b7-6129-4d28-bffc-95ef653cc648","Type":"ContainerDied","Data":"ae38028e00cba93eadd3aaaeab82cfd68221ebdd6e858869596c182258a7f71e"} Jul 09 02:13:32.614791 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:32.614462 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-669bbdcddd-f46ph" event={"ID":"2bab31b7-6129-4d28-bffc-95ef653cc648","Type":"ContainerDied","Data":"1dfa3685f9e7555962f1be4ff04584afe730b7d3bf5c85fa8fe1b18760658c31"} Jul 09 02:13:32.614791 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:32.614485 2582 scope.go:117] "RemoveContainer" containerID="ae38028e00cba93eadd3aaaeab82cfd68221ebdd6e858869596c182258a7f71e" Jul 09 02:13:32.623673 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:32.623655 2582 scope.go:117] "RemoveContainer" containerID="ae38028e00cba93eadd3aaaeab82cfd68221ebdd6e858869596c182258a7f71e" Jul 09 02:13:32.623950 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:13:32.623935 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae38028e00cba93eadd3aaaeab82cfd68221ebdd6e858869596c182258a7f71e\": container with ID starting with ae38028e00cba93eadd3aaaeab82cfd68221ebdd6e858869596c182258a7f71e not found: ID does not exist" containerID="ae38028e00cba93eadd3aaaeab82cfd68221ebdd6e858869596c182258a7f71e" Jul 09 02:13:32.624000 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:32.623965 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae38028e00cba93eadd3aaaeab82cfd68221ebdd6e858869596c182258a7f71e"} err="failed to get container status \"ae38028e00cba93eadd3aaaeab82cfd68221ebdd6e858869596c182258a7f71e\": rpc error: code = NotFound desc = could not find container \"ae38028e00cba93eadd3aaaeab82cfd68221ebdd6e858869596c182258a7f71e\": container with ID starting with ae38028e00cba93eadd3aaaeab82cfd68221ebdd6e858869596c182258a7f71e not found: ID does not exist" Jul 09 02:13:32.637719 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:32.637696 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-669bbdcddd-f46ph"] Jul 09 02:13:32.642099 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:32.642080 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-669bbdcddd-f46ph"] Jul 09 02:13:33.281913 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.281860 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:13:33.393729 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.393700 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-669bbdcddd-9cp7q"] Jul 09 02:13:33.394217 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.394197 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bab31b7-6129-4d28-bffc-95ef653cc648" containerName="manager" Jul 09 02:13:33.394217 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.394215 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bab31b7-6129-4d28-bffc-95ef653cc648" containerName="manager" Jul 09 02:13:33.394361 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.394329 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bab31b7-6129-4d28-bffc-95ef653cc648" containerName="manager" Jul 09 02:13:33.397299 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.397284 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-669bbdcddd-9cp7q" Jul 09 02:13:33.400479 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.400461 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 02:13:33.401905 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.401882 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m5h9z\"" Jul 09 02:13:33.402015 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.401963 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 02:13:33.407631 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.407601 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-669bbdcddd-9cp7q"] Jul 09 02:13:33.492633 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.492610 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28vjf\" (UniqueName: \"kubernetes.io/projected/27cf0c1a-9674-4132-b220-9bf4dc6834c1-kube-api-access-28vjf\") pod \"maas-controller-669bbdcddd-9cp7q\" (UID: \"27cf0c1a-9674-4132-b220-9bf4dc6834c1\") " pod="opendatahub/maas-controller-669bbdcddd-9cp7q" Jul 09 02:13:33.492776 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.492659 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27cf0c1a-9674-4132-b220-9bf4dc6834c1-webhook-cert\") pod \"maas-controller-669bbdcddd-9cp7q\" (UID: \"27cf0c1a-9674-4132-b220-9bf4dc6834c1\") " pod="opendatahub/maas-controller-669bbdcddd-9cp7q" Jul 09 02:13:33.593577 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.593519 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27cf0c1a-9674-4132-b220-9bf4dc6834c1-webhook-cert\") pod \"maas-controller-669bbdcddd-9cp7q\" (UID: \"27cf0c1a-9674-4132-b220-9bf4dc6834c1\") " pod="opendatahub/maas-controller-669bbdcddd-9cp7q" Jul 09 02:13:33.593577 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.593575 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28vjf\" (UniqueName: \"kubernetes.io/projected/27cf0c1a-9674-4132-b220-9bf4dc6834c1-kube-api-access-28vjf\") pod \"maas-controller-669bbdcddd-9cp7q\" (UID: \"27cf0c1a-9674-4132-b220-9bf4dc6834c1\") " pod="opendatahub/maas-controller-669bbdcddd-9cp7q" Jul 09 02:13:33.595821 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.595802 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27cf0c1a-9674-4132-b220-9bf4dc6834c1-webhook-cert\") pod \"maas-controller-669bbdcddd-9cp7q\" (UID: \"27cf0c1a-9674-4132-b220-9bf4dc6834c1\") " pod="opendatahub/maas-controller-669bbdcddd-9cp7q" Jul 09 02:13:33.603756 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.603735 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28vjf\" (UniqueName: \"kubernetes.io/projected/27cf0c1a-9674-4132-b220-9bf4dc6834c1-kube-api-access-28vjf\") pod \"maas-controller-669bbdcddd-9cp7q\" (UID: \"27cf0c1a-9674-4132-b220-9bf4dc6834c1\") " pod="opendatahub/maas-controller-669bbdcddd-9cp7q" Jul 09 02:13:33.707721 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.707692 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-669bbdcddd-9cp7q" Jul 09 02:13:33.836979 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.836956 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-669bbdcddd-9cp7q"] Jul 09 02:13:33.838953 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:13:33.838926 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27cf0c1a_9674_4132_b220_9bf4dc6834c1.slice/crio-ef7d1a936550ed00ceafc3425a384519ec3dcf418344a2c76541b32fc4a92fd7 WatchSource:0}: Error finding container ef7d1a936550ed00ceafc3425a384519ec3dcf418344a2c76541b32fc4a92fd7: Status 404 returned error can't find the container with id ef7d1a936550ed00ceafc3425a384519ec3dcf418344a2c76541b32fc4a92fd7 Jul 09 02:13:33.865035 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:33.864976 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bab31b7-6129-4d28-bffc-95ef653cc648" path="/var/lib/kubelet/pods/2bab31b7-6129-4d28-bffc-95ef653cc648/volumes" Jul 09 02:13:34.625935 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:34.625834 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-669bbdcddd-9cp7q" event={"ID":"27cf0c1a-9674-4132-b220-9bf4dc6834c1","Type":"ContainerStarted","Data":"fe0da98004c060af5ce20f17414d41933aaeb6b435530e872af2825eeea89827"} Jul 09 02:13:34.625935 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:34.625922 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-669bbdcddd-9cp7q" event={"ID":"27cf0c1a-9674-4132-b220-9bf4dc6834c1","Type":"ContainerStarted","Data":"ef7d1a936550ed00ceafc3425a384519ec3dcf418344a2c76541b32fc4a92fd7"} Jul 09 02:13:34.626111 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:34.626008 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-669bbdcddd-9cp7q" Jul 09 02:13:34.649796 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:34.649752 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-669bbdcddd-9cp7q" podStartSLOduration=1.167297014 podStartE2EDuration="1.64973693s" totalImagesPullingTime="482.439916ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:13:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:13:34.648340708 +0000 UTC m=+835.346394520" watchObservedRunningTime="2026-07-09 02:13:34.64973693 +0000 UTC m=+835.347790737" Jul 09 02:13:45.636712 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:45.636684 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-669bbdcddd-9cp7q" Jul 09 02:13:49.339028 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:49.338994 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:13:49.440184 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:49.440141 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:13:55.387284 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:55.387252 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:13:56.994607 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:56.994565 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:13:57.100666 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:13:57.100635 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:14:06.289650 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:14:06.289612 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:14:11.917119 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:14:11.917057 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:14:12.015142 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:14:12.015106 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:14:12.141917 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:14:12.141884 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:14:18.750306 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:14:18.750263 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:14:18.860924 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:14:18.860895 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:14:18.967023 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:14:18.966988 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:14:19.148745 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:14:19.148717 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:14:25.359248 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:14:25.359207 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:14:25.458912 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:14:25.458854 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:14:39.763507 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:14:39.763444 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:14:39.767955 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:14:39.767934 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:14:40.186388 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:14:40.186359 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:14:41.502504 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:14:41.502470 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:14:51.586813 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:14:51.586784 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:15:00.592159 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:15:00.592130 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:15:09.884160 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:15:09.884127 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:15:18.591470 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:15:18.591435 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:15:23.178651 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:15:23.178622 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:15:24.466640 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:15:24.466607 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:15:28.085039 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:15:28.085004 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:15:38.584647 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:15:38.584613 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:15:44.269022 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:15:44.268990 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:15:44.368594 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:15:44.368566 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:15:48.482513 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:15:48.482480 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:16:11.094343 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:16:11.094263 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:16:20.283704 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:16:20.283661 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:16:42.893258 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:16:42.893227 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:16:54.583797 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:16:54.583763 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:17:09.990565 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:17:09.990536 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:17:19.188645 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:17:19.188614 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:17:28.090866 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:17:28.090816 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:17:36.995236 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:17:36.995157 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:17:46.898004 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:17:46.897967 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:18:13.787224 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:18:13.787187 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh"] Jul 09 02:18:13.787767 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:18:13.787395 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh" podUID="35bd079f-3913-46ac-8ac3-d71cf740a453" containerName="manager" containerID="cri-o://192c31167ded0f69047f3339f11a6fd11a5288396149ec53f33d45f3fa2f51b6" gracePeriod=10 Jul 09 02:18:17.034074 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:18:17.034034 2582 generic.go:356] "Generic (PLEG): container finished" podID="35bd079f-3913-46ac-8ac3-d71cf740a453" containerID="192c31167ded0f69047f3339f11a6fd11a5288396149ec53f33d45f3fa2f51b6" exitCode=0 Jul 09 02:18:17.034444 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:18:17.034127 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh" event={"ID":"35bd079f-3913-46ac-8ac3-d71cf740a453","Type":"ContainerDied","Data":"192c31167ded0f69047f3339f11a6fd11a5288396149ec53f33d45f3fa2f51b6"} Jul 09 02:18:17.081220 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:18:17.081196 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh" Jul 09 02:18:17.201000 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:18:17.200911 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tngn7\" (UniqueName: \"kubernetes.io/projected/35bd079f-3913-46ac-8ac3-d71cf740a453-kube-api-access-tngn7\") pod \"35bd079f-3913-46ac-8ac3-d71cf740a453\" (UID: \"35bd079f-3913-46ac-8ac3-d71cf740a453\") " Jul 09 02:18:17.201605 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:18:17.201582 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35bd079f-3913-46ac-8ac3-d71cf740a453-extensions-socket-volume\") pod \"35bd079f-3913-46ac-8ac3-d71cf740a453\" (UID: \"35bd079f-3913-46ac-8ac3-d71cf740a453\") " Jul 09 02:18:17.202222 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:18:17.202194 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35bd079f-3913-46ac-8ac3-d71cf740a453-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "35bd079f-3913-46ac-8ac3-d71cf740a453" (UID: "35bd079f-3913-46ac-8ac3-d71cf740a453"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:18:17.202514 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:18:17.202496 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35bd079f-3913-46ac-8ac3-d71cf740a453-extensions-socket-volume\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:18:17.205211 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:18:17.205172 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35bd079f-3913-46ac-8ac3-d71cf740a453-kube-api-access-tngn7" (OuterVolumeSpecName: "kube-api-access-tngn7") pod "35bd079f-3913-46ac-8ac3-d71cf740a453" (UID: "35bd079f-3913-46ac-8ac3-d71cf740a453"). InnerVolumeSpecName "kube-api-access-tngn7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:18:17.303403 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:18:17.303375 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tngn7\" (UniqueName: \"kubernetes.io/projected/35bd079f-3913-46ac-8ac3-d71cf740a453-kube-api-access-tngn7\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:18:18.044014 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:18:18.043983 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh" event={"ID":"35bd079f-3913-46ac-8ac3-d71cf740a453","Type":"ContainerDied","Data":"09061796292dd2ad4335e02edbcac56fcb03c3d1d25a56301f6b34e000a4cebf"} Jul 09 02:18:18.044014 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:18:18.044004 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh" Jul 09 02:18:18.044436 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:18:18.044054 2582 scope.go:117] "RemoveContainer" containerID="192c31167ded0f69047f3339f11a6fd11a5288396149ec53f33d45f3fa2f51b6" Jul 09 02:18:18.065985 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:18:18.065958 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh"] Jul 09 02:18:18.070811 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:18:18.070787 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzrlh"] Jul 09 02:18:19.865357 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:18:19.865329 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35bd079f-3913-46ac-8ac3-d71cf740a453" path="/var/lib/kubelet/pods/35bd079f-3913-46ac-8ac3-d71cf740a453/volumes" Jul 09 02:19:14.975018 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:14.974937 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsbcx"] Jul 09 02:19:14.975494 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:14.975315 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35bd079f-3913-46ac-8ac3-d71cf740a453" containerName="manager" Jul 09 02:19:14.975494 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:14.975329 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="35bd079f-3913-46ac-8ac3-d71cf740a453" containerName="manager" Jul 09 02:19:14.975494 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:14.975402 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="35bd079f-3913-46ac-8ac3-d71cf740a453" containerName="manager" Jul 09 02:19:14.978223 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:14.978208 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsbcx" Jul 09 02:19:14.981182 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:14.981161 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f78gw\"" Jul 09 02:19:14.990001 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:14.989980 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsbcx"] Jul 09 02:19:15.054469 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:15.054440 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtgnc\" (UniqueName: \"kubernetes.io/projected/c7f67406-2d81-48fa-8c0f-d3855f65eddf-kube-api-access-wtgnc\") pod \"kuadrant-operator-controller-manager-86878c769f-rsbcx\" (UID: \"c7f67406-2d81-48fa-8c0f-d3855f65eddf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsbcx" Jul 09 02:19:15.054625 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:15.054520 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7f67406-2d81-48fa-8c0f-d3855f65eddf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rsbcx\" (UID: \"c7f67406-2d81-48fa-8c0f-d3855f65eddf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsbcx" Jul 09 02:19:15.154978 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:15.154946 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtgnc\" (UniqueName: \"kubernetes.io/projected/c7f67406-2d81-48fa-8c0f-d3855f65eddf-kube-api-access-wtgnc\") pod \"kuadrant-operator-controller-manager-86878c769f-rsbcx\" (UID: \"c7f67406-2d81-48fa-8c0f-d3855f65eddf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsbcx" Jul 09 02:19:15.155118 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:15.155069 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7f67406-2d81-48fa-8c0f-d3855f65eddf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rsbcx\" (UID: \"c7f67406-2d81-48fa-8c0f-d3855f65eddf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsbcx" Jul 09 02:19:15.155379 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:15.155364 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7f67406-2d81-48fa-8c0f-d3855f65eddf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rsbcx\" (UID: \"c7f67406-2d81-48fa-8c0f-d3855f65eddf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsbcx" Jul 09 02:19:15.166151 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:15.166123 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtgnc\" (UniqueName: \"kubernetes.io/projected/c7f67406-2d81-48fa-8c0f-d3855f65eddf-kube-api-access-wtgnc\") pod \"kuadrant-operator-controller-manager-86878c769f-rsbcx\" (UID: \"c7f67406-2d81-48fa-8c0f-d3855f65eddf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsbcx" Jul 09 02:19:15.288845 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:15.288771 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsbcx" Jul 09 02:19:15.426712 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:15.426690 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsbcx"] Jul 09 02:19:15.428523 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:19:15.428497 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7f67406_2d81_48fa_8c0f_d3855f65eddf.slice/crio-d96fa4eab6bc37519645b1b16bd6ce96c6c13fed4ba9b6c492296846f05289b1 WatchSource:0}: Error finding container d96fa4eab6bc37519645b1b16bd6ce96c6c13fed4ba9b6c492296846f05289b1: Status 404 returned error can't find the container with id d96fa4eab6bc37519645b1b16bd6ce96c6c13fed4ba9b6c492296846f05289b1 Jul 09 02:19:16.328530 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:16.328493 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsbcx" event={"ID":"c7f67406-2d81-48fa-8c0f-d3855f65eddf","Type":"ContainerStarted","Data":"1468e14fb3ee63b213a4b4ba45b5b814c31cd1426971d4f1f1b623ff7b06610b"} Jul 09 02:19:16.328913 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:16.328542 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsbcx" event={"ID":"c7f67406-2d81-48fa-8c0f-d3855f65eddf","Type":"ContainerStarted","Data":"d96fa4eab6bc37519645b1b16bd6ce96c6c13fed4ba9b6c492296846f05289b1"} Jul 09 02:19:16.328913 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:16.328614 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsbcx" Jul 09 02:19:16.354084 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:16.354031 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsbcx" podStartSLOduration=2.354014619 podStartE2EDuration="2.354014619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:19:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:19:16.352712745 +0000 UTC m=+1177.050766560" watchObservedRunningTime="2026-07-09 02:19:16.354014619 +0000 UTC m=+1177.052068431" Jul 09 02:19:27.337741 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:27.337658 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsbcx" Jul 09 02:19:34.489300 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:34.489264 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:19:35.071310 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:35.071283 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:19:35.163176 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:35.163146 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:19:39.794821 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:39.794789 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:19:39.797537 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:39.797514 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:19:44.279343 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:44.279311 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:19:55.391695 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:19:55.391656 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:20:06.088597 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:20:06.088563 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:20:06.965136 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:20:06.965103 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:20:16.185960 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:20:16.185925 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:20:26.584579 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:20:26.584543 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:20:36.491733 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:20:36.491635 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:20:40.237625 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:20:40.237595 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:20:47.282146 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:20:47.282108 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:20:58.189978 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:20:58.189942 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:21:06.892933 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:21:06.892895 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:21:17.486765 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:21:17.486734 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:21:41.689080 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:21:41.689043 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:21:45.752465 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:21:45.752427 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:21:45.852214 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:21:45.852182 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:21:48.976089 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:21:48.976048 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:21:58.188306 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:21:58.188274 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:22:10.278378 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:22:10.278305 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:22:18.177319 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:22:18.177287 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:22:28.489559 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:22:28.489526 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:22:38.685405 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:22:38.685371 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:22:47.982199 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:22:47.982167 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:22:56.683548 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:22:56.683523 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:23:07.090074 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:23:07.090036 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:23:16.488321 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:23:16.488286 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:23:29.683120 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:23:29.683086 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:23:38.888766 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:23:38.888683 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:23:48.282934 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:23:48.282893 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:23:59.776794 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:23:59.776762 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:24:12.382179 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:24:12.382146 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:24:20.378917 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:24:20.378860 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:24:30.882002 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:24:30.881967 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:24:39.837633 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:24:39.837603 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:24:39.838135 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:24:39.837810 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:24:40.687933 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:24:40.687895 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:24:52.081335 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:24:52.081301 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:24:55.535024 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:24:55.534986 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:03.617005 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:03.616923 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:03.678028 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:03.677992 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:04.784886 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:04.784845 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:25:05.186433 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:05.186385 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:06.735168 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:06.735131 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:15.036435 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:15.036398 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:15.587045 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:15.587009 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:25:16.751276 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:16.751243 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:20.060473 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:20.060437 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:22.857382 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:22.857340 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:22.957553 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:22.957526 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:23.245131 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:23.245097 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:24.290920 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:24.290891 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:25:29.255032 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:29.255002 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:29.355958 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:29.355919 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:29.455231 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:29.455201 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:29.542070 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:29.541985 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:30.551888 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:30.551844 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:30.644086 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:30.644053 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:30.714784 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:30.714753 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:31.102271 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:31.102236 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:31.204682 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:31.204647 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:31.303945 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:31.303912 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:33.422103 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:33.422068 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:33.685064 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:33.685031 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:25:35.699811 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:35.699754 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:35.902772 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:35.902738 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:36.003453 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:36.003382 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:36.077904 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:36.077854 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:37.455205 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:37.455174 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:41.980717 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:41.980674 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:42.193037 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:42.193006 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:25:43.394340 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:43.394312 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:43.505301 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:43.505269 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:43.873640 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:43.873612 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:44.011988 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:44.011956 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:44.106205 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:44.106173 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:45.379336 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:45.379281 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:48.354281 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:48.354252 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:48.777470 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:48.777437 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:54.438501 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:54.438468 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:54.535842 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:54.535806 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:25:56.191538 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:25:56.191513 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:00.423221 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:00.423190 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:00.526530 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:00.526504 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:00.626190 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:00.626151 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:00.730490 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:00.730408 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:01.807262 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:01.807213 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:01.889646 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:01.889613 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:01.964644 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:01.964611 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:02.319308 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:02.319272 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:02.466728 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:02.466662 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:02.565205 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:02.565157 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:03.925063 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:03.925037 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:06.157144 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:06.157112 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:06.280731 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:06.280693 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:26:06.348670 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:06.348641 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:06.480101 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:06.480017 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:06.565897 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:06.565851 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:10.154179 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:10.154146 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:12.411855 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:12.411812 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:12.472765 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:12.472740 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:12.877226 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:12.877149 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:12.970283 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:12.970255 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:13.069203 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:13.069177 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:13.085768 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:13.085742 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:26:14.420704 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:14.420656 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:20.416242 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:20.416202 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:20.517075 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:20.517040 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:20.614555 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:20.614502 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:20.715514 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:20.715437 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:21.103163 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:21.103078 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:47.047653 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:47.047613 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:53.089586 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:53.089550 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:53.158698 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:53.158670 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:26:53.238844 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:26:53.238811 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:09.153754 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:09.153722 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:15.195921 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:15.195891 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:15.258829 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:15.258802 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:15.341120 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:15.341087 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:15.439259 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:15.439230 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:25.918610 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:25.918574 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:26.692679 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:26.692634 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:32.725938 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:32.725905 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:32.788368 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:32.788338 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:32.859592 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:32.859558 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:32.960152 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:32.960120 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:33.059856 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:33.059772 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:33.734969 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:33.734932 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:33.840654 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:33.840624 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:33.948653 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:33.948622 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:35.524229 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:35.524191 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:35.985812 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:35.985784 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:27:41.503430 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:41.503403 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:41.565274 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:41.565243 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:42.165497 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:42.165468 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:45.281457 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:45.281424 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:27:48.281671 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:48.281645 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:48.381405 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:48.381378 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:48.501748 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:48.501723 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:48.802785 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:48.802752 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:49.194848 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:49.194818 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:51.594223 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:51.594194 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:51.694939 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:51.694901 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:51.831118 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:51.831079 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:51.939380 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:51.939351 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:52.330479 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:52.330397 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:52.427917 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:52.427862 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:56.101563 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:56.101529 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:56.586274 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:56.586248 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:27:56.724428 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:56.724390 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:56.826346 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:56.826318 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:57.160581 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:57.160553 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:27:59.952931 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:27:59.952901 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:02.999399 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:02.999308 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:03.099705 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:03.099676 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:03.416141 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:03.416119 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:03.775864 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:03.775789 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:03.857662 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:03.857630 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:08.081827 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:08.081794 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:28:09.954949 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:09.954915 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:10.020374 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:10.020343 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:10.216079 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:10.215998 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:10.440049 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:10.440020 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:10.544587 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:10.544484 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:10.647610 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:10.647577 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:10.768412 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:10.768375 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:16.727539 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:16.727512 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:16.791717 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:16.791680 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:16.867592 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:16.867540 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:16.969979 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:16.969948 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:17.073400 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:17.073311 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:23.169545 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:23.169515 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:23.237412 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:23.237383 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:29.799934 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:29.799902 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:30.105085 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:30.105007 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:30.309103 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:30.309069 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:32.817915 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:32.817860 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:32.904392 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:32.904367 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:32.968713 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:32.968685 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:33.268981 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:33.268949 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:33.378147 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:33.378115 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:33.482853 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:33.482827 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:37.355137 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:37.355106 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:37.985020 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:37.984988 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:28:41.781465 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:41.781436 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr"] Jul 09 02:28:41.784700 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:41.784676 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:41.787922 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:41.787898 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7d9bdee4\"/\"kube-root-ca.crt\"" Jul 09 02:28:41.789727 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:41.789708 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7d9bdee4\"/\"e2e-lifecycle-model-7d9bdee4-kserve-self-signed-certs\"" Jul 09 02:28:41.789852 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:41.789725 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7d9bdee4\"/\"default-dockercfg-st9bn\"" Jul 09 02:28:41.789852 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:41.789783 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7d9bdee4\"/\"openshift-service-ca.crt\"" Jul 09 02:28:41.793713 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:41.793693 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr"] Jul 09 02:28:41.812858 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:41.812835 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:41.910520 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:41.910492 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-dshm\") pod \"e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:41.910675 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:41.910536 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-tmp-dir\") pod \"e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:41.910675 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:41.910643 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71e0589e-3422-4eb1-b83a-db198c7b2cfe-tls-certs\") pod \"e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:41.910772 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:41.910676 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-home\") pod \"e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:41.910772 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:41.910705 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-model-cache\") pod \"e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:41.910772 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:41.910726 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lmhq\" (UniqueName: \"kubernetes.io/projected/71e0589e-3422-4eb1-b83a-db198c7b2cfe-kube-api-access-7lmhq\") pod \"e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:42.012118 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:42.012090 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71e0589e-3422-4eb1-b83a-db198c7b2cfe-tls-certs\") pod \"e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:42.012118 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:42.012122 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-home\") pod \"e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:42.012355 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:42.012146 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-model-cache\") pod \"e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:42.012355 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:42.012167 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lmhq\" (UniqueName: \"kubernetes.io/projected/71e0589e-3422-4eb1-b83a-db198c7b2cfe-kube-api-access-7lmhq\") pod \"e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:42.012355 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:42.012207 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-dshm\") pod \"e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:42.012355 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:42.012248 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-tmp-dir\") pod \"e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:42.012626 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:42.012606 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-home\") pod \"e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:42.012684 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:42.012643 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-model-cache\") pod \"e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:42.012684 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:42.012658 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-tmp-dir\") pod \"e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:42.014526 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:42.014505 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-dshm\") pod \"e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:42.014767 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:42.014751 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71e0589e-3422-4eb1-b83a-db198c7b2cfe-tls-certs\") pod \"e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:42.020094 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:42.020072 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lmhq\" (UniqueName: \"kubernetes.io/projected/71e0589e-3422-4eb1-b83a-db198c7b2cfe-kube-api-access-7lmhq\") pod \"e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:42.096184 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:42.096111 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:42.241142 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:42.240952 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr"] Jul 09 02:28:43.167474 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:43.167443 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" event={"ID":"71e0589e-3422-4eb1-b83a-db198c7b2cfe","Type":"ContainerStarted","Data":"ed2b11a093e0d2e34a68b8f3a98ea9bb8caf4f5202ed1feb7f37d968a88b1c17"} Jul 09 02:28:43.167840 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:43.167483 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" event={"ID":"71e0589e-3422-4eb1-b83a-db198c7b2cfe","Type":"ContainerStarted","Data":"2ffa0ef1276da6cce6b8dd06e2ab0c8065b998f670914e1409cc7da4aad05598"} Jul 09 02:28:43.186058 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:43.186014 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" podStartSLOduration=2.185999592 podStartE2EDuration="2.185999592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:28:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:28:43.184072846 +0000 UTC m=+1743.882126663" watchObservedRunningTime="2026-07-09 02:28:43.185999592 +0000 UTC m=+1743.884053405" Jul 09 02:28:46.984972 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:46.984938 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:28:52.096706 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:52.096672 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:52.097097 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:52.096737 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:52.109122 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:52.109098 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:52.220523 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:52.220496 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:28:53.643035 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:53.643003 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:53.743717 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:53.743687 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:54.463856 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:54.463826 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:54.564349 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:54.564317 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:56.885819 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:56.885790 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:28:59.484979 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:59.484940 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:28:59.894112 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:28:59.894036 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:29:05.541672 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:05.541637 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:29:09.773601 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:09.773564 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:29:10.518158 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:10.518119 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr"] Jul 09 02:29:10.567324 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:10.567295 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:29:25.541891 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.541837 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" podUID="71e0589e-3422-4eb1-b83a-db198c7b2cfe" containerName="main" containerID="cri-o://ed2b11a093e0d2e34a68b8f3a98ea9bb8caf4f5202ed1feb7f37d968a88b1c17" gracePeriod=45 Jul 09 02:29:25.783387 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.783365 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:29:25.876889 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.876804 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-model-cache\") pod \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " Jul 09 02:29:25.876889 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.876838 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-home\") pod \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " Jul 09 02:29:25.877093 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.876909 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lmhq\" (UniqueName: \"kubernetes.io/projected/71e0589e-3422-4eb1-b83a-db198c7b2cfe-kube-api-access-7lmhq\") pod \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " Jul 09 02:29:25.877093 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.876930 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-tmp-dir\") pod \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " Jul 09 02:29:25.877093 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.876951 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71e0589e-3422-4eb1-b83a-db198c7b2cfe-tls-certs\") pod \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " Jul 09 02:29:25.877093 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.877009 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-dshm\") pod \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\" (UID: \"71e0589e-3422-4eb1-b83a-db198c7b2cfe\") " Jul 09 02:29:25.877093 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.877049 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-model-cache" (OuterVolumeSpecName: "model-cache") pod "71e0589e-3422-4eb1-b83a-db198c7b2cfe" (UID: "71e0589e-3422-4eb1-b83a-db198c7b2cfe"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:29:25.877320 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.877230 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-model-cache\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:29:25.877320 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.877231 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-home" (OuterVolumeSpecName: "home") pod "71e0589e-3422-4eb1-b83a-db198c7b2cfe" (UID: "71e0589e-3422-4eb1-b83a-db198c7b2cfe"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:29:25.877320 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.877242 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "71e0589e-3422-4eb1-b83a-db198c7b2cfe" (UID: "71e0589e-3422-4eb1-b83a-db198c7b2cfe"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:29:25.879011 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.878990 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-dshm" (OuterVolumeSpecName: "dshm") pod "71e0589e-3422-4eb1-b83a-db198c7b2cfe" (UID: "71e0589e-3422-4eb1-b83a-db198c7b2cfe"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:29:25.879083 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.879023 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71e0589e-3422-4eb1-b83a-db198c7b2cfe-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "71e0589e-3422-4eb1-b83a-db198c7b2cfe" (UID: "71e0589e-3422-4eb1-b83a-db198c7b2cfe"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 02:29:25.879128 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.879090 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71e0589e-3422-4eb1-b83a-db198c7b2cfe-kube-api-access-7lmhq" (OuterVolumeSpecName: "kube-api-access-7lmhq") pod "71e0589e-3422-4eb1-b83a-db198c7b2cfe" (UID: "71e0589e-3422-4eb1-b83a-db198c7b2cfe"). InnerVolumeSpecName "kube-api-access-7lmhq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:29:25.977592 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.977570 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-home\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:29:25.977592 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.977591 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lmhq\" (UniqueName: \"kubernetes.io/projected/71e0589e-3422-4eb1-b83a-db198c7b2cfe-kube-api-access-7lmhq\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:29:25.977726 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.977602 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-tmp-dir\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:29:25.977726 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.977610 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71e0589e-3422-4eb1-b83a-db198c7b2cfe-tls-certs\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:29:25.977726 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:25.977619 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71e0589e-3422-4eb1-b83a-db198c7b2cfe-dshm\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:29:26.380355 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:26.380322 2582 generic.go:356] "Generic (PLEG): container finished" podID="71e0589e-3422-4eb1-b83a-db198c7b2cfe" containerID="ed2b11a093e0d2e34a68b8f3a98ea9bb8caf4f5202ed1feb7f37d968a88b1c17" exitCode=0 Jul 09 02:29:26.380534 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:26.380371 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" Jul 09 02:29:26.380534 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:26.380380 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" event={"ID":"71e0589e-3422-4eb1-b83a-db198c7b2cfe","Type":"ContainerDied","Data":"ed2b11a093e0d2e34a68b8f3a98ea9bb8caf4f5202ed1feb7f37d968a88b1c17"} Jul 09 02:29:26.380534 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:26.380411 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr" event={"ID":"71e0589e-3422-4eb1-b83a-db198c7b2cfe","Type":"ContainerDied","Data":"2ffa0ef1276da6cce6b8dd06e2ab0c8065b998f670914e1409cc7da4aad05598"} Jul 09 02:29:26.380534 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:26.380429 2582 scope.go:117] "RemoveContainer" containerID="ed2b11a093e0d2e34a68b8f3a98ea9bb8caf4f5202ed1feb7f37d968a88b1c17" Jul 09 02:29:26.389767 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:26.389746 2582 scope.go:117] "RemoveContainer" containerID="ed2b11a093e0d2e34a68b8f3a98ea9bb8caf4f5202ed1feb7f37d968a88b1c17" Jul 09 02:29:26.390035 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:29:26.390020 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed2b11a093e0d2e34a68b8f3a98ea9bb8caf4f5202ed1feb7f37d968a88b1c17\": container with ID starting with ed2b11a093e0d2e34a68b8f3a98ea9bb8caf4f5202ed1feb7f37d968a88b1c17 not found: ID does not exist" containerID="ed2b11a093e0d2e34a68b8f3a98ea9bb8caf4f5202ed1feb7f37d968a88b1c17" Jul 09 02:29:26.390096 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:26.390042 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed2b11a093e0d2e34a68b8f3a98ea9bb8caf4f5202ed1feb7f37d968a88b1c17"} err="failed to get container status \"ed2b11a093e0d2e34a68b8f3a98ea9bb8caf4f5202ed1feb7f37d968a88b1c17\": rpc error: code = NotFound desc = could not find container \"ed2b11a093e0d2e34a68b8f3a98ea9bb8caf4f5202ed1feb7f37d968a88b1c17\": container with ID starting with ed2b11a093e0d2e34a68b8f3a98ea9bb8caf4f5202ed1feb7f37d968a88b1c17 not found: ID does not exist" Jul 09 02:29:26.404240 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:26.404215 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr"] Jul 09 02:29:26.406022 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:26.406002 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-7d9bdee4/e2e-lifecycle-model-7d9bdee4-kserve-59bc597b9f-fpwfr"] Jul 09 02:29:27.864933 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:27.864900 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71e0589e-3422-4eb1-b83a-db198c7b2cfe" path="/var/lib/kubelet/pods/71e0589e-3422-4eb1-b83a-db198c7b2cfe/volumes" Jul 09 02:29:39.880446 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:39.880418 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:29:39.880446 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:39.880442 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:29:47.140921 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:47.140864 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:29:47.855456 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:47.855375 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:29:50.479962 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:50.479933 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:29:53.913595 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:53.913561 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:29:53.979134 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:53.979105 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:29:54.186891 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:54.186794 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:29:54.413810 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:54.413785 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:29:55.471956 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:55.471926 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:29:55.571377 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:55.571344 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:29:55.672677 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:55.672645 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:29:56.363806 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:56.363759 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:29:56.462101 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:56.462058 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:29:56.562310 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:56.562279 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:29:57.919226 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:57.919191 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:29:59.877334 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:29:59.877304 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:30:03.765390 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:03.765357 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:03.831000 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:03.830967 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:04.428821 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:04.428788 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:10.375001 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:10.374971 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:10.483525 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:10.483484 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:10.575955 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:10.575923 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:11.984385 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:11.984350 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:30:21.061399 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:21.061367 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:21.400538 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:21.400507 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:42.342308 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:42.342277 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:48.237324 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:48.237292 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:49.573955 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:49.573920 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:51.551792 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:51.551734 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:51.653409 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:51.653372 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:51.748225 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:51.748190 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:52.102733 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:52.102698 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:52.204588 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:52.204547 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:52.303399 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:52.303370 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:56.040311 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.040235 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:56.363947 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.363841 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:56.642687 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.642614 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm"] Jul 09 02:30:56.643014 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.643000 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e0589e-3422-4eb1-b83a-db198c7b2cfe" containerName="main" Jul 09 02:30:56.643068 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.643017 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e0589e-3422-4eb1-b83a-db198c7b2cfe" containerName="main" Jul 09 02:30:56.643122 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.643112 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="71e0589e-3422-4eb1-b83a-db198c7b2cfe" containerName="main" Jul 09 02:30:56.646781 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.646212 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.649160 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.649135 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-5a89bf-openshift-default-dockercfg-8n2ms\"" Jul 09 02:30:56.649292 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.649251 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 02:30:56.658155 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.658130 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm"] Jul 09 02:30:56.670451 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.670431 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:56.786329 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.786299 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/412a92f6-ee69-4581-87a4-f723d980455b-istio-podinfo\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.786484 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.786339 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkhnh\" (UniqueName: \"kubernetes.io/projected/412a92f6-ee69-4581-87a4-f723d980455b-kube-api-access-tkhnh\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.786484 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.786367 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-credential-socket\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.786484 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.786419 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-workload-certs\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.786484 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.786463 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/412a92f6-ee69-4581-87a4-f723d980455b-istiod-ca-cert\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.786644 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.786497 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/412a92f6-ee69-4581-87a4-f723d980455b-istio-token\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.786644 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.786542 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-workload-socket\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.786644 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.786560 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-istio-data\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.786644 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.786595 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-istio-envoy\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.887702 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.887669 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-credential-socket\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.887702 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.887700 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-workload-certs\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.887956 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.887725 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/412a92f6-ee69-4581-87a4-f723d980455b-istiod-ca-cert\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.887956 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.887752 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/412a92f6-ee69-4581-87a4-f723d980455b-istio-token\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.887956 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.887851 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-workload-socket\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.887956 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.887889 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-istio-data\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.887956 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.887924 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-istio-envoy\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.888169 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.887964 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/412a92f6-ee69-4581-87a4-f723d980455b-istio-podinfo\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.888169 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.888116 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-credential-socket\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.888169 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.888146 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkhnh\" (UniqueName: \"kubernetes.io/projected/412a92f6-ee69-4581-87a4-f723d980455b-kube-api-access-tkhnh\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.888335 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.888315 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-istio-data\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.888395 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.888385 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-workload-certs\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.888448 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.888414 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-workload-socket\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.888666 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.888641 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/412a92f6-ee69-4581-87a4-f723d980455b-istiod-ca-cert\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.890236 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.890172 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-istio-envoy\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.890507 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.890490 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/412a92f6-ee69-4581-87a4-f723d980455b-istio-podinfo\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.895514 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.895460 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/412a92f6-ee69-4581-87a4-f723d980455b-istio-token\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.896330 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.896310 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkhnh\" (UniqueName: \"kubernetes.io/projected/412a92f6-ee69-4581-87a4-f723d980455b-kube-api-access-tkhnh\") pod \"e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:56.920857 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.920834 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:30:56.957001 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:56.956976 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:30:57.093568 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:57.093546 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm"] Jul 09 02:30:57.095304 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:30:57.095277 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod412a92f6_ee69_4581_87a4_f723d980455b.slice/crio-c75cf18dede97a619cc6ff2576d98d83279666c4f4f18570e344b2a363368dda WatchSource:0}: Error finding container c75cf18dede97a619cc6ff2576d98d83279666c4f4f18570e344b2a363368dda: Status 404 returned error can't find the container with id c75cf18dede97a619cc6ff2576d98d83279666c4f4f18570e344b2a363368dda Jul 09 02:30:57.097732 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:57.097717 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 02:30:57.840960 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:57.840925 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" event={"ID":"412a92f6-ee69-4581-87a4-f723d980455b","Type":"ContainerStarted","Data":"c75cf18dede97a619cc6ff2576d98d83279666c4f4f18570e344b2a363368dda"} Jul 09 02:30:59.948083 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:59.948021 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:30:59.948390 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:59.948117 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:30:59.948390 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:30:59.948148 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:31:00.282474 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:00.282386 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:31:00.858528 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:00.858492 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" event={"ID":"412a92f6-ee69-4581-87a4-f723d980455b","Type":"ContainerStarted","Data":"c01887b80d5b707ef50e8f1d6bbaf1f8ecfafca75a96effd4a95e85da1deeea4"} Jul 09 02:31:00.878312 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:00.878267 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" podStartSLOduration=2.028356416 podStartE2EDuration="4.87825152s" totalImagesPullingTime="2.849895104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:31:00.877404388 +0000 UTC m=+1881.575458202" watchObservedRunningTime="2026-07-09 02:31:00.87825152 +0000 UTC m=+1881.576305399" Jul 09 02:31:00.957644 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:00.957620 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:31:01.962423 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:01.962396 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:31:02.748031 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:02.747997 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:31:02.833751 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:02.833714 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:31:02.869264 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:02.869235 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:31:02.870218 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:02.870199 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:31:02.897905 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:02.897852 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:31:03.458627 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:03.458594 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:31:03.558286 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:03.558255 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:31:07.245337 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:07.245299 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:31:12.834675 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:12.834597 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:31:12.936354 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:12.936319 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:31:14.509157 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:14.509083 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:31:14.929537 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:14.929505 2582 generic.go:356] "Generic (PLEG): container finished" podID="412a92f6-ee69-4581-87a4-f723d980455b" containerID="c01887b80d5b707ef50e8f1d6bbaf1f8ecfafca75a96effd4a95e85da1deeea4" exitCode=0 Jul 09 02:31:14.929714 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:14.929575 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" event={"ID":"412a92f6-ee69-4581-87a4-f723d980455b","Type":"ContainerDied","Data":"c01887b80d5b707ef50e8f1d6bbaf1f8ecfafca75a96effd4a95e85da1deeea4"} Jul 09 02:31:14.930042 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:14.930025 2582 scope.go:117] "RemoveContainer" containerID="c01887b80d5b707ef50e8f1d6bbaf1f8ecfafca75a96effd4a95e85da1deeea4" Jul 09 02:31:14.931147 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:14.931115 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:31:14.931267 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:14.931199 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:31:14.931267 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:14.931249 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:31:15.939697 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:15.939656 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" event={"ID":"412a92f6-ee69-4581-87a4-f723d980455b","Type":"ContainerStarted","Data":"10bc3143b0d843236db70e5b336565b351390e8a93fb95a0e28c8c81bff586df"} Jul 09 02:31:15.958184 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:15.958151 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:31:15.959348 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:15.959317 2582 patch_prober.go:28] interesting pod/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 09 02:31:15.959471 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:15.959370 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" podUID="412a92f6-ee69-4581-87a4-f723d980455b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 09 02:31:16.957989 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:16.957520 2582 patch_prober.go:28] interesting pod/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 09 02:31:16.957989 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:16.957598 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" podUID="412a92f6-ee69-4581-87a4-f723d980455b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 09 02:31:17.869604 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:17.869571 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:31:17.958350 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:17.958309 2582 patch_prober.go:28] interesting pod/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 09 02:31:17.958809 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:17.958378 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" podUID="412a92f6-ee69-4581-87a4-f723d980455b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 09 02:31:18.957659 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:18.957625 2582 patch_prober.go:28] interesting pod/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 09 02:31:18.957993 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:18.957946 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" podUID="412a92f6-ee69-4581-87a4-f723d980455b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 09 02:31:19.961911 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:19.961864 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:31:19.963089 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:19.963067 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:31:21.415074 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:21.415044 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:31:24.120180 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:24.120144 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:31:24.222949 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:24.222915 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:31:25.983318 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:25.983286 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:31:39.301629 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:39.301594 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:31:46.383345 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:31:46.383310 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:32:00.681703 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:00.681667 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:32:05.697920 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:05.697860 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:06.628684 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:06.628645 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:06.824647 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:06.824611 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:07.223088 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:07.223056 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:07.324864 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:07.324834 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:07.464738 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:07.464706 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:12.318740 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:12.318698 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:17.767240 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:17.761972 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:32:49.193750 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:49.193668 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:49.478347 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:49.478270 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:49.710587 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:49.710554 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:49.890425 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:49.890344 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:50.352725 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:50.352671 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:50.452859 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:50.452821 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:50.553056 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:50.553023 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:50.688651 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:50.688619 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:52.582171 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:52.582138 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:32:56.438740 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:56.438703 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:56.539381 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:56.539336 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:56.670148 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:56.670114 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm"] Jul 09 02:32:56.670354 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:56.670331 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" podUID="412a92f6-ee69-4581-87a4-f723d980455b" containerName="istio-proxy" containerID="cri-o://10bc3143b0d843236db70e5b336565b351390e8a93fb95a0e28c8c81bff586df" gracePeriod=30 Jul 09 02:32:56.681989 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:56.681958 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:56.908782 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:56.908754 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:57.238078 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:57.237990 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:57.542137 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:57.542055 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:32:57.659094 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:32:57.659067 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:00.213956 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:00.213919 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:00.285889 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:00.285847 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:00.347721 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:00.347694 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:00.465411 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:00.465323 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:00.833805 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:00.833727 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:00.934338 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:00.934301 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:01.923829 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:01.923772 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:33:02.050860 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.050776 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-credential-socket\") pod \"412a92f6-ee69-4581-87a4-f723d980455b\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " Jul 09 02:33:02.050860 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.050807 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-istio-data\") pod \"412a92f6-ee69-4581-87a4-f723d980455b\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " Jul 09 02:33:02.050860 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.050827 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkhnh\" (UniqueName: \"kubernetes.io/projected/412a92f6-ee69-4581-87a4-f723d980455b-kube-api-access-tkhnh\") pod \"412a92f6-ee69-4581-87a4-f723d980455b\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " Jul 09 02:33:02.051170 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.050863 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/412a92f6-ee69-4581-87a4-f723d980455b-istiod-ca-cert\") pod \"412a92f6-ee69-4581-87a4-f723d980455b\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " Jul 09 02:33:02.051170 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.050910 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-istio-envoy\") pod \"412a92f6-ee69-4581-87a4-f723d980455b\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " Jul 09 02:33:02.051170 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.050952 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/412a92f6-ee69-4581-87a4-f723d980455b-istio-podinfo\") pod \"412a92f6-ee69-4581-87a4-f723d980455b\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " Jul 09 02:33:02.051170 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.050984 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-workload-certs\") pod \"412a92f6-ee69-4581-87a4-f723d980455b\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " Jul 09 02:33:02.051170 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.051007 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-workload-socket\") pod \"412a92f6-ee69-4581-87a4-f723d980455b\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " Jul 09 02:33:02.051170 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.051043 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/412a92f6-ee69-4581-87a4-f723d980455b-istio-token\") pod \"412a92f6-ee69-4581-87a4-f723d980455b\" (UID: \"412a92f6-ee69-4581-87a4-f723d980455b\") " Jul 09 02:33:02.051170 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.051143 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "412a92f6-ee69-4581-87a4-f723d980455b" (UID: "412a92f6-ee69-4581-87a4-f723d980455b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:33:02.051509 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.051277 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "412a92f6-ee69-4581-87a4-f723d980455b" (UID: "412a92f6-ee69-4581-87a4-f723d980455b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:33:02.051509 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.051358 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "412a92f6-ee69-4581-87a4-f723d980455b" (UID: "412a92f6-ee69-4581-87a4-f723d980455b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:33:02.051509 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.051386 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-credential-socket\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:33:02.051509 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.051382 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/412a92f6-ee69-4581-87a4-f723d980455b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "412a92f6-ee69-4581-87a4-f723d980455b" (UID: "412a92f6-ee69-4581-87a4-f723d980455b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 02:33:02.051509 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.051401 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-workload-certs\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:33:02.052306 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.052277 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-istio-data" (OuterVolumeSpecName: "istio-data") pod "412a92f6-ee69-4581-87a4-f723d980455b" (UID: "412a92f6-ee69-4581-87a4-f723d980455b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:33:02.053233 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.053210 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/412a92f6-ee69-4581-87a4-f723d980455b-kube-api-access-tkhnh" (OuterVolumeSpecName: "kube-api-access-tkhnh") pod "412a92f6-ee69-4581-87a4-f723d980455b" (UID: "412a92f6-ee69-4581-87a4-f723d980455b"). InnerVolumeSpecName "kube-api-access-tkhnh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:33:02.053322 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.053226 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "412a92f6-ee69-4581-87a4-f723d980455b" (UID: "412a92f6-ee69-4581-87a4-f723d980455b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:33:02.053322 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.053292 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/412a92f6-ee69-4581-87a4-f723d980455b-istio-token" (OuterVolumeSpecName: "istio-token") pod "412a92f6-ee69-4581-87a4-f723d980455b" (UID: "412a92f6-ee69-4581-87a4-f723d980455b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:33:02.053322 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.053287 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/412a92f6-ee69-4581-87a4-f723d980455b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "412a92f6-ee69-4581-87a4-f723d980455b" (UID: "412a92f6-ee69-4581-87a4-f723d980455b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 02:33:02.152274 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.152233 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-istio-data\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:33:02.152274 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.152270 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkhnh\" (UniqueName: \"kubernetes.io/projected/412a92f6-ee69-4581-87a4-f723d980455b-kube-api-access-tkhnh\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:33:02.152468 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.152286 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/412a92f6-ee69-4581-87a4-f723d980455b-istiod-ca-cert\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:33:02.152468 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.152300 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-istio-envoy\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:33:02.152468 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.152312 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/412a92f6-ee69-4581-87a4-f723d980455b-istio-podinfo\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:33:02.152468 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.152323 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/412a92f6-ee69-4581-87a4-f723d980455b-workload-socket\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:33:02.152468 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.152336 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/412a92f6-ee69-4581-87a4-f723d980455b-istio-token\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:33:02.492463 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.492431 2582 generic.go:356] "Generic (PLEG): container finished" podID="412a92f6-ee69-4581-87a4-f723d980455b" containerID="10bc3143b0d843236db70e5b336565b351390e8a93fb95a0e28c8c81bff586df" exitCode=0 Jul 09 02:33:02.492669 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.492486 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" Jul 09 02:33:02.492669 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.492487 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" event={"ID":"412a92f6-ee69-4581-87a4-f723d980455b","Type":"ContainerDied","Data":"10bc3143b0d843236db70e5b336565b351390e8a93fb95a0e28c8c81bff586df"} Jul 09 02:33:02.492669 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.492614 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm" event={"ID":"412a92f6-ee69-4581-87a4-f723d980455b","Type":"ContainerDied","Data":"c75cf18dede97a619cc6ff2576d98d83279666c4f4f18570e344b2a363368dda"} Jul 09 02:33:02.492669 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.492643 2582 scope.go:117] "RemoveContainer" containerID="10bc3143b0d843236db70e5b336565b351390e8a93fb95a0e28c8c81bff586df" Jul 09 02:33:02.502214 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.502197 2582 scope.go:117] "RemoveContainer" containerID="c01887b80d5b707ef50e8f1d6bbaf1f8ecfafca75a96effd4a95e85da1deeea4" Jul 09 02:33:02.510233 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.510217 2582 scope.go:117] "RemoveContainer" containerID="10bc3143b0d843236db70e5b336565b351390e8a93fb95a0e28c8c81bff586df" Jul 09 02:33:02.510511 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:33:02.510493 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10bc3143b0d843236db70e5b336565b351390e8a93fb95a0e28c8c81bff586df\": container with ID starting with 10bc3143b0d843236db70e5b336565b351390e8a93fb95a0e28c8c81bff586df not found: ID does not exist" containerID="10bc3143b0d843236db70e5b336565b351390e8a93fb95a0e28c8c81bff586df" Jul 09 02:33:02.510550 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.510520 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10bc3143b0d843236db70e5b336565b351390e8a93fb95a0e28c8c81bff586df"} err="failed to get container status \"10bc3143b0d843236db70e5b336565b351390e8a93fb95a0e28c8c81bff586df\": rpc error: code = NotFound desc = could not find container \"10bc3143b0d843236db70e5b336565b351390e8a93fb95a0e28c8c81bff586df\": container with ID starting with 10bc3143b0d843236db70e5b336565b351390e8a93fb95a0e28c8c81bff586df not found: ID does not exist" Jul 09 02:33:02.510550 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.510537 2582 scope.go:117] "RemoveContainer" containerID="c01887b80d5b707ef50e8f1d6bbaf1f8ecfafca75a96effd4a95e85da1deeea4" Jul 09 02:33:02.510762 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:33:02.510744 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c01887b80d5b707ef50e8f1d6bbaf1f8ecfafca75a96effd4a95e85da1deeea4\": container with ID starting with c01887b80d5b707ef50e8f1d6bbaf1f8ecfafca75a96effd4a95e85da1deeea4 not found: ID does not exist" containerID="c01887b80d5b707ef50e8f1d6bbaf1f8ecfafca75a96effd4a95e85da1deeea4" Jul 09 02:33:02.510798 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.510768 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c01887b80d5b707ef50e8f1d6bbaf1f8ecfafca75a96effd4a95e85da1deeea4"} err="failed to get container status \"c01887b80d5b707ef50e8f1d6bbaf1f8ecfafca75a96effd4a95e85da1deeea4\": rpc error: code = NotFound desc = could not find container \"c01887b80d5b707ef50e8f1d6bbaf1f8ecfafca75a96effd4a95e85da1deeea4\": container with ID starting with c01887b80d5b707ef50e8f1d6bbaf1f8ecfafca75a96effd4a95e85da1deeea4 not found: ID does not exist" Jul 09 02:33:02.516238 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.516215 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm"] Jul 09 02:33:02.519477 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:02.519458 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-5a89bf-openshift-default-687b6cf969-r82sm"] Jul 09 02:33:03.865760 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:03.865729 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="412a92f6-ee69-4581-87a4-f723d980455b" path="/var/lib/kubelet/pods/412a92f6-ee69-4581-87a4-f723d980455b/volumes" Jul 09 02:33:04.738261 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:04.738221 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:05.187167 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.187132 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:05.471130 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.471053 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw"] Jul 09 02:33:05.471631 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.471614 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="412a92f6-ee69-4581-87a4-f723d980455b" containerName="istio-proxy" Jul 09 02:33:05.471631 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.471632 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="412a92f6-ee69-4581-87a4-f723d980455b" containerName="istio-proxy" Jul 09 02:33:05.471743 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.471643 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="412a92f6-ee69-4581-87a4-f723d980455b" containerName="istio-proxy" Jul 09 02:33:05.471743 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.471648 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="412a92f6-ee69-4581-87a4-f723d980455b" containerName="istio-proxy" Jul 09 02:33:05.471743 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.471716 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="412a92f6-ee69-4581-87a4-f723d980455b" containerName="istio-proxy" Jul 09 02:33:05.476533 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.476517 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.479340 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.479319 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-f9c58b-openshift-default-dockercfg-dd4cl\"" Jul 09 02:33:05.479553 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.479536 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 02:33:05.484101 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.484079 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw"] Jul 09 02:33:05.490384 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.490364 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:05.578325 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.578297 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-credential-socket\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.578495 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.578336 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-workload-certs\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.578495 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.578360 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-istio-data\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.578495 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.578402 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3704dbb-0170-459e-acdf-d90c3604a394-istio-token\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.578495 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.578437 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3704dbb-0170-459e-acdf-d90c3604a394-istio-podinfo\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.578495 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.578469 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3704dbb-0170-459e-acdf-d90c3604a394-istiod-ca-cert\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.578764 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.578566 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-istio-envoy\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.578764 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.578605 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8fmq\" (UniqueName: \"kubernetes.io/projected/e3704dbb-0170-459e-acdf-d90c3604a394-kube-api-access-g8fmq\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.578764 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.578669 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-workload-socket\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.679369 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.679338 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-istio-data\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.679369 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.679370 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3704dbb-0170-459e-acdf-d90c3604a394-istio-token\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.679599 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.679386 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3704dbb-0170-459e-acdf-d90c3604a394-istio-podinfo\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.679599 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.679404 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3704dbb-0170-459e-acdf-d90c3604a394-istiod-ca-cert\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.679599 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.679483 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-istio-envoy\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.679599 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.679513 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8fmq\" (UniqueName: \"kubernetes.io/projected/e3704dbb-0170-459e-acdf-d90c3604a394-kube-api-access-g8fmq\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.679599 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.679550 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-workload-socket\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.679895 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.679611 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-credential-socket\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.679895 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.679635 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-workload-certs\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.679895 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.679790 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-istio-data\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.680033 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.680012 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-workload-socket\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.680092 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.680029 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-workload-certs\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.680092 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.680045 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-credential-socket\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.680345 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.680324 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3704dbb-0170-459e-acdf-d90c3604a394-istiod-ca-cert\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.681757 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.681736 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-istio-envoy\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.682265 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.682245 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3704dbb-0170-459e-acdf-d90c3604a394-istio-podinfo\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.688025 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.687990 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8fmq\" (UniqueName: \"kubernetes.io/projected/e3704dbb-0170-459e-acdf-d90c3604a394-kube-api-access-g8fmq\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.688197 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.688176 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3704dbb-0170-459e-acdf-d90c3604a394-istio-token\") pod \"e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.766342 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.766274 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:05.788461 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.788442 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:05.926653 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.926631 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw"] Jul 09 02:33:05.928405 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:33:05.928382 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3704dbb_0170_459e_acdf_d90c3604a394.slice/crio-2fccb8f7e88dd06f1c793b9d0ac342a30f88d046f49b4bedbb3e5991a76f0b15 WatchSource:0}: Error finding container 2fccb8f7e88dd06f1c793b9d0ac342a30f88d046f49b4bedbb3e5991a76f0b15: Status 404 returned error can't find the container with id 2fccb8f7e88dd06f1c793b9d0ac342a30f88d046f49b4bedbb3e5991a76f0b15 Jul 09 02:33:05.930596 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.930562 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:33:05.930715 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.930637 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:33:05.930715 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:05.930697 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:33:06.516192 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:06.516158 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" event={"ID":"e3704dbb-0170-459e-acdf-d90c3604a394","Type":"ContainerStarted","Data":"e4bd84f58fda8028dfd0ab7c9a3d0b584dcbf8947446c4b7f9f8c62e7e7db8a1"} Jul 09 02:33:06.516562 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:06.516234 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" event={"ID":"e3704dbb-0170-459e-acdf-d90c3604a394","Type":"ContainerStarted","Data":"2fccb8f7e88dd06f1c793b9d0ac342a30f88d046f49b4bedbb3e5991a76f0b15"} Jul 09 02:33:06.539028 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:06.538978 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" podStartSLOduration=1.538961784 podStartE2EDuration="1.538961784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:33:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:33:06.536346051 +0000 UTC m=+2007.234399865" watchObservedRunningTime="2026-07-09 02:33:06.538961784 +0000 UTC m=+2007.237015597" Jul 09 02:33:06.788978 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:06.788889 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:06.794306 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:06.794284 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:07.180670 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:07.180639 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:33:07.521675 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:07.521604 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:07.522506 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:07.522487 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:08.394439 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:08.394409 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:08.459369 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:08.459333 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:08.519395 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:08.519363 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:08.683792 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:08.683761 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:09.046411 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:09.046336 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:09.148673 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:09.148638 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:12.887859 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:12.887827 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:19.808830 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:19.808795 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:19.910094 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:19.910066 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:20.887942 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:20.887888 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:21.291647 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:21.291614 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:21.391669 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:21.391639 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:21.533330 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:21.533299 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:23.415775 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:23.415743 2582 patch_prober.go:28] interesting pod/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": read tcp 10.134.0.2:38798->10.134.0.46:15021: read: connection reset by peer" start-of-body= Jul 09 02:33:23.416165 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:23.415810 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" podUID="e3704dbb-0170-459e-acdf-d90c3604a394" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": read tcp 10.134.0.2:38798->10.134.0.46:15021: read: connection reset by peer" Jul 09 02:33:23.607724 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:23.607648 2582 generic.go:356] "Generic (PLEG): container finished" podID="e3704dbb-0170-459e-acdf-d90c3604a394" containerID="e4bd84f58fda8028dfd0ab7c9a3d0b584dcbf8947446c4b7f9f8c62e7e7db8a1" exitCode=0 Jul 09 02:33:23.607847 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:23.607726 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" event={"ID":"e3704dbb-0170-459e-acdf-d90c3604a394","Type":"ContainerDied","Data":"e4bd84f58fda8028dfd0ab7c9a3d0b584dcbf8947446c4b7f9f8c62e7e7db8a1"} Jul 09 02:33:23.608136 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:23.608122 2582 scope.go:117] "RemoveContainer" containerID="e4bd84f58fda8028dfd0ab7c9a3d0b584dcbf8947446c4b7f9f8c62e7e7db8a1" Jul 09 02:33:23.609072 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:23.609047 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:33:23.609126 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:23.609113 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:33:23.609169 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:23.609144 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:33:24.584929 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:24.584899 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:33:24.614671 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:24.614640 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" event={"ID":"e3704dbb-0170-459e-acdf-d90c3604a394","Type":"ContainerStarted","Data":"7059fbf0d85629b995e174d437510e278c0071795de7f9b64d371223e98c7d27"} Jul 09 02:33:24.789673 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:24.789630 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:24.789853 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:24.789733 2582 patch_prober.go:28] interesting pod/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 09 02:33:24.789853 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:24.789777 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" podUID="e3704dbb-0170-459e-acdf-d90c3604a394" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 09 02:33:25.789154 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:25.789118 2582 patch_prober.go:28] interesting pod/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 09 02:33:25.789552 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:25.789176 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" podUID="e3704dbb-0170-459e-acdf-d90c3604a394" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 09 02:33:26.789505 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:26.789471 2582 patch_prober.go:28] interesting pod/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 09 02:33:26.789966 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:26.789524 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" podUID="e3704dbb-0170-459e-acdf-d90c3604a394" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 09 02:33:27.108074 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:27.107978 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:27.208095 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:27.208065 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:27.366957 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:27.362452 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:27.586431 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:27.586397 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:27.789697 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:27.789667 2582 patch_prober.go:28] interesting pod/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 09 02:33:27.790074 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:27.789719 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" podUID="e3704dbb-0170-459e-acdf-d90c3604a394" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 09 02:33:27.805724 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:27.805690 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:28.228238 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:28.228204 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:28.486932 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:28.486824 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:28.587928 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:28.587898 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:28.747400 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:28.747320 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:28.789266 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:28.789227 2582 patch_prober.go:28] interesting pod/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 09 02:33:28.789446 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:28.789287 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" podUID="e3704dbb-0170-459e-acdf-d90c3604a394" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 09 02:33:29.793106 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:29.793076 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:29.793483 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:29.793326 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:29.794078 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:29.794063 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:34.116703 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:34.116674 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:34.217022 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:34.216993 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:34.360261 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:34.360229 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw"] Jul 09 02:33:34.360433 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:34.360412 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" podUID="e3704dbb-0170-459e-acdf-d90c3604a394" containerName="istio-proxy" containerID="cri-o://7059fbf0d85629b995e174d437510e278c0071795de7f9b64d371223e98c7d27" gracePeriod=30 Jul 09 02:33:34.373238 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:34.373178 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:34.591302 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:34.591268 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:34.917471 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:34.917444 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:35.206224 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:35.206147 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:35.917485 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:35.917452 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:38.282918 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:38.282885 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:33:38.438438 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:38.438380 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:38.509083 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:38.509052 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:38.561768 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:38.561687 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:38.760503 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:38.760472 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:38.975354 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:38.975322 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:39.075918 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.075885 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:39.608505 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.608485 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:39.676784 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.676758 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3704dbb-0170-459e-acdf-d90c3604a394-istiod-ca-cert\") pod \"e3704dbb-0170-459e-acdf-d90c3604a394\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " Jul 09 02:33:39.676784 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.676786 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-workload-socket\") pod \"e3704dbb-0170-459e-acdf-d90c3604a394\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " Jul 09 02:33:39.677040 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.676808 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3704dbb-0170-459e-acdf-d90c3604a394-istio-podinfo\") pod \"e3704dbb-0170-459e-acdf-d90c3604a394\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " Jul 09 02:33:39.677040 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.676835 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-credential-socket\") pod \"e3704dbb-0170-459e-acdf-d90c3604a394\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " Jul 09 02:33:39.677040 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.676897 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-istio-data\") pod \"e3704dbb-0170-459e-acdf-d90c3604a394\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " Jul 09 02:33:39.677040 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.676932 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-workload-certs\") pod \"e3704dbb-0170-459e-acdf-d90c3604a394\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " Jul 09 02:33:39.677040 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.676958 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-istio-envoy\") pod \"e3704dbb-0170-459e-acdf-d90c3604a394\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " Jul 09 02:33:39.677040 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.676982 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8fmq\" (UniqueName: \"kubernetes.io/projected/e3704dbb-0170-459e-acdf-d90c3604a394-kube-api-access-g8fmq\") pod \"e3704dbb-0170-459e-acdf-d90c3604a394\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " Jul 09 02:33:39.677040 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.677028 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3704dbb-0170-459e-acdf-d90c3604a394-istio-token\") pod \"e3704dbb-0170-459e-acdf-d90c3604a394\" (UID: \"e3704dbb-0170-459e-acdf-d90c3604a394\") " Jul 09 02:33:39.677040 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.677027 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e3704dbb-0170-459e-acdf-d90c3604a394" (UID: "e3704dbb-0170-459e-acdf-d90c3604a394"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:33:39.677459 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.677207 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e3704dbb-0170-459e-acdf-d90c3604a394" (UID: "e3704dbb-0170-459e-acdf-d90c3604a394"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:33:39.677459 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.677257 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3704dbb-0170-459e-acdf-d90c3604a394-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e3704dbb-0170-459e-acdf-d90c3604a394" (UID: "e3704dbb-0170-459e-acdf-d90c3604a394"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 02:33:39.677459 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.677375 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e3704dbb-0170-459e-acdf-d90c3604a394" (UID: "e3704dbb-0170-459e-acdf-d90c3604a394"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:33:39.677459 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.677422 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3704dbb-0170-459e-acdf-d90c3604a394-istiod-ca-cert\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:33:39.677459 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.677440 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-workload-socket\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:33:39.677459 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.677452 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-workload-certs\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:33:39.677940 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.677918 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-istio-data" (OuterVolumeSpecName: "istio-data") pod "e3704dbb-0170-459e-acdf-d90c3604a394" (UID: "e3704dbb-0170-459e-acdf-d90c3604a394"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:33:39.679067 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.679039 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3704dbb-0170-459e-acdf-d90c3604a394-kube-api-access-g8fmq" (OuterVolumeSpecName: "kube-api-access-g8fmq") pod "e3704dbb-0170-459e-acdf-d90c3604a394" (UID: "e3704dbb-0170-459e-acdf-d90c3604a394"). InnerVolumeSpecName "kube-api-access-g8fmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:33:39.679163 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.679122 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e3704dbb-0170-459e-acdf-d90c3604a394-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e3704dbb-0170-459e-acdf-d90c3604a394" (UID: "e3704dbb-0170-459e-acdf-d90c3604a394"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 02:33:39.679414 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.679395 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e3704dbb-0170-459e-acdf-d90c3604a394" (UID: "e3704dbb-0170-459e-acdf-d90c3604a394"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:33:39.679477 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.679461 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3704dbb-0170-459e-acdf-d90c3604a394-istio-token" (OuterVolumeSpecName: "istio-token") pod "e3704dbb-0170-459e-acdf-d90c3604a394" (UID: "e3704dbb-0170-459e-acdf-d90c3604a394"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:33:39.696007 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.695982 2582 generic.go:356] "Generic (PLEG): container finished" podID="e3704dbb-0170-459e-acdf-d90c3604a394" containerID="7059fbf0d85629b995e174d437510e278c0071795de7f9b64d371223e98c7d27" exitCode=0 Jul 09 02:33:39.696090 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.696039 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" Jul 09 02:33:39.696090 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.696064 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" event={"ID":"e3704dbb-0170-459e-acdf-d90c3604a394","Type":"ContainerDied","Data":"7059fbf0d85629b995e174d437510e278c0071795de7f9b64d371223e98c7d27"} Jul 09 02:33:39.696160 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.696119 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw" event={"ID":"e3704dbb-0170-459e-acdf-d90c3604a394","Type":"ContainerDied","Data":"2fccb8f7e88dd06f1c793b9d0ac342a30f88d046f49b4bedbb3e5991a76f0b15"} Jul 09 02:33:39.696160 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.696146 2582 scope.go:117] "RemoveContainer" containerID="7059fbf0d85629b995e174d437510e278c0071795de7f9b64d371223e98c7d27" Jul 09 02:33:39.704844 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.704827 2582 scope.go:117] "RemoveContainer" containerID="e4bd84f58fda8028dfd0ab7c9a3d0b584dcbf8947446c4b7f9f8c62e7e7db8a1" Jul 09 02:33:39.712725 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.712709 2582 scope.go:117] "RemoveContainer" containerID="7059fbf0d85629b995e174d437510e278c0071795de7f9b64d371223e98c7d27" Jul 09 02:33:39.712970 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:33:39.712951 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7059fbf0d85629b995e174d437510e278c0071795de7f9b64d371223e98c7d27\": container with ID starting with 7059fbf0d85629b995e174d437510e278c0071795de7f9b64d371223e98c7d27 not found: ID does not exist" containerID="7059fbf0d85629b995e174d437510e278c0071795de7f9b64d371223e98c7d27" Jul 09 02:33:39.713041 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.712980 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7059fbf0d85629b995e174d437510e278c0071795de7f9b64d371223e98c7d27"} err="failed to get container status \"7059fbf0d85629b995e174d437510e278c0071795de7f9b64d371223e98c7d27\": rpc error: code = NotFound desc = could not find container \"7059fbf0d85629b995e174d437510e278c0071795de7f9b64d371223e98c7d27\": container with ID starting with 7059fbf0d85629b995e174d437510e278c0071795de7f9b64d371223e98c7d27 not found: ID does not exist" Jul 09 02:33:39.713041 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.712999 2582 scope.go:117] "RemoveContainer" containerID="e4bd84f58fda8028dfd0ab7c9a3d0b584dcbf8947446c4b7f9f8c62e7e7db8a1" Jul 09 02:33:39.713214 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:33:39.713194 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4bd84f58fda8028dfd0ab7c9a3d0b584dcbf8947446c4b7f9f8c62e7e7db8a1\": container with ID starting with e4bd84f58fda8028dfd0ab7c9a3d0b584dcbf8947446c4b7f9f8c62e7e7db8a1 not found: ID does not exist" containerID="e4bd84f58fda8028dfd0ab7c9a3d0b584dcbf8947446c4b7f9f8c62e7e7db8a1" Jul 09 02:33:39.713265 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.713223 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4bd84f58fda8028dfd0ab7c9a3d0b584dcbf8947446c4b7f9f8c62e7e7db8a1"} err="failed to get container status \"e4bd84f58fda8028dfd0ab7c9a3d0b584dcbf8947446c4b7f9f8c62e7e7db8a1\": rpc error: code = NotFound desc = could not find container \"e4bd84f58fda8028dfd0ab7c9a3d0b584dcbf8947446c4b7f9f8c62e7e7db8a1\": container with ID starting with e4bd84f58fda8028dfd0ab7c9a3d0b584dcbf8947446c4b7f9f8c62e7e7db8a1 not found: ID does not exist" Jul 09 02:33:39.720642 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.720623 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw"] Jul 09 02:33:39.724114 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.724095 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-f9c58b-openshift-default-8484c848dc-mwwzw"] Jul 09 02:33:39.778217 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.778192 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3704dbb-0170-459e-acdf-d90c3604a394-istio-podinfo\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:33:39.778217 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.778213 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-credential-socket\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:33:39.778361 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.778224 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-istio-data\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:33:39.778361 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.778233 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3704dbb-0170-459e-acdf-d90c3604a394-istio-envoy\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:33:39.778361 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.778241 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8fmq\" (UniqueName: \"kubernetes.io/projected/e3704dbb-0170-459e-acdf-d90c3604a394-kube-api-access-g8fmq\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:33:39.778361 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.778249 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3704dbb-0170-459e-acdf-d90c3604a394-istio-token\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:33:39.871691 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:39.871665 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3704dbb-0170-459e-acdf-d90c3604a394" path="/var/lib/kubelet/pods/e3704dbb-0170-459e-acdf-d90c3604a394/volumes" Jul 09 02:33:42.940840 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:42.940805 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:43.251914 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.251817 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:43.507538 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.507457 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr"] Jul 09 02:33:43.507811 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.507797 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3704dbb-0170-459e-acdf-d90c3604a394" containerName="istio-proxy" Jul 09 02:33:43.507902 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.507811 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3704dbb-0170-459e-acdf-d90c3604a394" containerName="istio-proxy" Jul 09 02:33:43.507902 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.507898 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3704dbb-0170-459e-acdf-d90c3604a394" containerName="istio-proxy" Jul 09 02:33:43.508016 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.507909 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3704dbb-0170-459e-acdf-d90c3604a394" containerName="istio-proxy" Jul 09 02:33:43.508016 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.507916 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="412a92f6-ee69-4581-87a4-f723d980455b" containerName="istio-proxy" Jul 09 02:33:43.509831 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.509813 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.512826 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.512806 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-c6afad-openshift-default-dockercfg-2sr57\"" Jul 09 02:33:43.512971 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.512846 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 02:33:43.521179 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.521157 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr"] Jul 09 02:33:43.544575 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.544546 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:43.610503 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.610475 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-workload-certs\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.610629 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.610513 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-credential-socket\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.610629 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.610540 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-podinfo\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.610629 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.610621 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/935cd0ff-9d88-4db0-9856-fd50f784a542-istiod-ca-cert\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.610754 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.610653 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-envoy\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.610754 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.610677 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-data\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.610754 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.610731 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-workload-socket\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.610754 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.610751 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-token\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.610927 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.610766 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjdvw\" (UniqueName: \"kubernetes.io/projected/935cd0ff-9d88-4db0-9856-fd50f784a542-kube-api-access-fjdvw\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.711257 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.711226 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/935cd0ff-9d88-4db0-9856-fd50f784a542-istiod-ca-cert\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.711433 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.711268 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-envoy\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.711433 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.711288 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-data\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.711433 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.711315 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-workload-socket\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.711433 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.711333 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-token\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.711433 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.711349 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjdvw\" (UniqueName: \"kubernetes.io/projected/935cd0ff-9d88-4db0-9856-fd50f784a542-kube-api-access-fjdvw\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.711433 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.711394 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-workload-certs\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.711774 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.711696 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-credential-socket\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.711774 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.711735 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-data\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.711774 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.711759 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-podinfo\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.711969 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.711812 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-workload-certs\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.711969 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.711925 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-workload-socket\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.712075 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.712052 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/935cd0ff-9d88-4db0-9856-fd50f784a542-istiod-ca-cert\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.712127 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.712106 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-credential-socket\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.714077 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.714059 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-podinfo\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.714204 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.714130 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-envoy\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.719448 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.719424 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-token\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.719718 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.719696 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjdvw\" (UniqueName: \"kubernetes.io/projected/935cd0ff-9d88-4db0-9856-fd50f784a542-kube-api-access-fjdvw\") pod \"e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.800331 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.800230 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:43.822062 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.822034 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:43.956081 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.956059 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr"] Jul 09 02:33:43.957862 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:33:43.957833 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod935cd0ff_9d88_4db0_9856_fd50f784a542.slice/crio-99c533008328f174ac3ae48b113274d63d57ed7629d8fe5a8cc6adf208077b39 WatchSource:0}: Error finding container 99c533008328f174ac3ae48b113274d63d57ed7629d8fe5a8cc6adf208077b39: Status 404 returned error can't find the container with id 99c533008328f174ac3ae48b113274d63d57ed7629d8fe5a8cc6adf208077b39 Jul 09 02:33:43.960421 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.960388 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:33:43.960536 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.960450 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:33:43.960536 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:43.960480 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:33:44.723926 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:44.723894 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" event={"ID":"935cd0ff-9d88-4db0-9856-fd50f784a542","Type":"ContainerStarted","Data":"04b8ba949996b1f6e05f2da3b8930331bd48b0af35ee8df25c66691128bc01bc"} Jul 09 02:33:44.724069 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:44.723948 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" event={"ID":"935cd0ff-9d88-4db0-9856-fd50f784a542","Type":"ContainerStarted","Data":"99c533008328f174ac3ae48b113274d63d57ed7629d8fe5a8cc6adf208077b39"} Jul 09 02:33:44.747072 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:44.747020 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" podStartSLOduration=1.747003146 podStartE2EDuration="1.747003146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:33:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:33:44.745667748 +0000 UTC m=+2045.443721571" watchObservedRunningTime="2026-07-09 02:33:44.747003146 +0000 UTC m=+2045.445056959" Jul 09 02:33:44.822357 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:44.822329 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:45.827182 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:45.827155 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:46.713079 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:46.713040 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:46.739701 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:46.739670 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:46.740660 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:46.740629 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:46.767176 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:46.767151 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:46.899972 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:46.899944 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:47.185040 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:47.185008 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:47.446070 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:47.445993 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:47.548079 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:47.548047 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:49.979547 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:49.979512 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:33:51.237802 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:51.237770 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:54.323052 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:54.323017 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:54.423280 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:54.423248 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:57.803527 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:57.803496 2582 generic.go:356] "Generic (PLEG): container finished" podID="935cd0ff-9d88-4db0-9856-fd50f784a542" containerID="04b8ba949996b1f6e05f2da3b8930331bd48b0af35ee8df25c66691128bc01bc" exitCode=0 Jul 09 02:33:57.803972 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:57.803576 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" event={"ID":"935cd0ff-9d88-4db0-9856-fd50f784a542","Type":"ContainerDied","Data":"04b8ba949996b1f6e05f2da3b8930331bd48b0af35ee8df25c66691128bc01bc"} Jul 09 02:33:57.804024 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:57.803984 2582 scope.go:117] "RemoveContainer" containerID="04b8ba949996b1f6e05f2da3b8930331bd48b0af35ee8df25c66691128bc01bc" Jul 09 02:33:57.805039 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:57.805015 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:33:57.805115 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:57.805080 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:33:57.805115 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:57.805110 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 02:33:58.810847 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:58.810811 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" event={"ID":"935cd0ff-9d88-4db0-9856-fd50f784a542","Type":"ContainerStarted","Data":"f4dba92fc804cf9c2f2990732bd69fb1ebc6c969c2f4dafa54751f2c977fff90"} Jul 09 02:33:58.822712 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:58.822685 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:33:58.823862 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:58.823833 2582 patch_prober.go:28] interesting pod/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 09 02:33:58.823956 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:58.823903 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" podUID="935cd0ff-9d88-4db0-9856-fd50f784a542" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 09 02:33:59.781753 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:59.781722 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:33:59.822965 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:59.822935 2582 patch_prober.go:28] interesting pod/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 09 02:33:59.823333 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:59.822986 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" podUID="935cd0ff-9d88-4db0-9856-fd50f784a542" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 09 02:33:59.878888 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:33:59.878844 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:34:00.822324 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:00.822290 2582 patch_prober.go:28] interesting pod/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 09 02:34:00.822487 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:00.822340 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" podUID="935cd0ff-9d88-4db0-9856-fd50f784a542" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 09 02:34:01.428360 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:01.428326 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:34:01.740299 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:01.740199 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:34:01.823228 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:01.823194 2582 patch_prober.go:28] interesting pod/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 09 02:34:01.823427 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:01.823267 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" podUID="935cd0ff-9d88-4db0-9856-fd50f784a542" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 09 02:34:02.823002 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:02.822970 2582 patch_prober.go:28] interesting pod/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 09 02:34:02.823382 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:02.823021 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" podUID="935cd0ff-9d88-4db0-9856-fd50f784a542" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 09 02:34:03.826570 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:03.826537 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:34:03.827638 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:03.827610 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:34:04.577745 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:04.577710 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:34:08.112067 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:08.112026 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:34:08.211166 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:08.211132 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:34:28.922330 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:28.922301 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:34:32.328163 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:32.328127 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:34:32.385017 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:32.384984 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:34:32.386041 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:32.386020 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:34:39.911910 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:39.911864 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:34:39.914503 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:39.914484 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:34:50.187043 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:34:50.187008 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:35:08.084890 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:35:08.084840 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:35:24.187102 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:35:24.187057 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:35:42.589786 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:35:42.589706 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:35:57.783151 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:35:57.783116 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:36:16.795044 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:16.795014 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:36:27.584914 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:27.584850 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:36:28.933391 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:28.933356 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:36:33.074102 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.074070 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk"] Jul 09 02:36:33.074460 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.074421 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3704dbb-0170-459e-acdf-d90c3604a394" containerName="istio-proxy" Jul 09 02:36:33.074460 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.074432 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3704dbb-0170-459e-acdf-d90c3604a394" containerName="istio-proxy" Jul 09 02:36:33.076559 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.076542 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.079492 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.079469 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7124ed\"/\"default-dockercfg-krmq7\"" Jul 09 02:36:33.081239 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.081216 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7124ed\"/\"sub-test-model-7124ed-kserve-self-signed-certs\"" Jul 09 02:36:33.081352 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.081216 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7124ed\"/\"kube-root-ca.crt\"" Jul 09 02:36:33.081352 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.081235 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7124ed\"/\"openshift-service-ca.crt\"" Jul 09 02:36:33.086579 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.086557 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk"] Jul 09 02:36:33.101219 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.101194 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:36:33.110215 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.110194 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-tmp-dir\") pod \"sub-test-model-7124ed-kserve-6f759bdf57-rg8zk\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.110315 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.110239 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv84b\" (UniqueName: \"kubernetes.io/projected/8534267e-58e2-459e-8332-1660b15ee85f-kube-api-access-wv84b\") pod \"sub-test-model-7124ed-kserve-6f759bdf57-rg8zk\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.110315 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.110304 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-dshm\") pod \"sub-test-model-7124ed-kserve-6f759bdf57-rg8zk\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.110417 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.110371 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-model-cache\") pod \"sub-test-model-7124ed-kserve-6f759bdf57-rg8zk\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.110417 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.110413 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8534267e-58e2-459e-8332-1660b15ee85f-tls-certs\") pod \"sub-test-model-7124ed-kserve-6f759bdf57-rg8zk\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.110499 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.110448 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-home\") pod \"sub-test-model-7124ed-kserve-6f759bdf57-rg8zk\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.211333 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.211302 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-tmp-dir\") pod \"sub-test-model-7124ed-kserve-6f759bdf57-rg8zk\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.211458 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.211341 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv84b\" (UniqueName: \"kubernetes.io/projected/8534267e-58e2-459e-8332-1660b15ee85f-kube-api-access-wv84b\") pod \"sub-test-model-7124ed-kserve-6f759bdf57-rg8zk\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.211458 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.211366 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-dshm\") pod \"sub-test-model-7124ed-kserve-6f759bdf57-rg8zk\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.211458 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.211390 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-model-cache\") pod \"sub-test-model-7124ed-kserve-6f759bdf57-rg8zk\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.211458 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.211415 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8534267e-58e2-459e-8332-1660b15ee85f-tls-certs\") pod \"sub-test-model-7124ed-kserve-6f759bdf57-rg8zk\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.211458 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.211442 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-home\") pod \"sub-test-model-7124ed-kserve-6f759bdf57-rg8zk\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.211759 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.211740 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-tmp-dir\") pod \"sub-test-model-7124ed-kserve-6f759bdf57-rg8zk\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.211818 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.211802 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-model-cache\") pod \"sub-test-model-7124ed-kserve-6f759bdf57-rg8zk\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.211896 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.211842 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-home\") pod \"sub-test-model-7124ed-kserve-6f759bdf57-rg8zk\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.213548 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.213531 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-dshm\") pod \"sub-test-model-7124ed-kserve-6f759bdf57-rg8zk\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.213802 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.213786 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8534267e-58e2-459e-8332-1660b15ee85f-tls-certs\") pod \"sub-test-model-7124ed-kserve-6f759bdf57-rg8zk\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.219205 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.219182 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv84b\" (UniqueName: \"kubernetes.io/projected/8534267e-58e2-459e-8332-1660b15ee85f-kube-api-access-wv84b\") pod \"sub-test-model-7124ed-kserve-6f759bdf57-rg8zk\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.386484 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.386398 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:33.523125 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.523099 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk"] Jul 09 02:36:33.524532 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:36:33.524510 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8534267e_58e2_459e_8332_1660b15ee85f.slice/crio-d03920dec9691fa723bf6f2cd68169583eac65c81ad3f6c081ce05cbd8bd2aa8 WatchSource:0}: Error finding container d03920dec9691fa723bf6f2cd68169583eac65c81ad3f6c081ce05cbd8bd2aa8: Status 404 returned error can't find the container with id d03920dec9691fa723bf6f2cd68169583eac65c81ad3f6c081ce05cbd8bd2aa8 Jul 09 02:36:33.595092 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.595058 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" event={"ID":"8534267e-58e2-459e-8332-1660b15ee85f","Type":"ContainerStarted","Data":"4cdae972e1ad8058ae59da03dbf2a03e797a28c8b8ac41fafbd6b9d284fce06e"} Jul 09 02:36:33.595219 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.595099 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" event={"ID":"8534267e-58e2-459e-8332-1660b15ee85f","Type":"ContainerStarted","Data":"d03920dec9691fa723bf6f2cd68169583eac65c81ad3f6c081ce05cbd8bd2aa8"} Jul 09 02:36:33.613373 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.613325 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" podStartSLOduration=0.613308709 podStartE2EDuration="613.308709ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:36:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:36:33.61104981 +0000 UTC m=+2214.309103624" watchObservedRunningTime="2026-07-09 02:36:33.613308709 +0000 UTC m=+2214.311362523" Jul 09 02:36:33.778594 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:33.778558 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:36:43.386928 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:43.386900 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:43.387327 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:43.386960 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:43.399130 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:43.399107 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:43.660304 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:43.660278 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:36:46.579811 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:46.579775 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:36:54.734926 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:54.734892 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:36:56.611510 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:36:56.611476 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:37:01.486642 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:37:01.486539 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:37:22.789252 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:37:22.789215 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:37:41.984853 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:37:41.984821 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:37:47.094863 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:37:47.094828 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk"] Jul 09 02:37:47.112638 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:37:47.112596 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:37:48.021003 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:37:48.020970 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:38:00.180646 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:00.180612 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:38:02.116029 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.115966 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" podUID="8534267e-58e2-459e-8332-1660b15ee85f" containerName="main" containerID="cri-o://4cdae972e1ad8058ae59da03dbf2a03e797a28c8b8ac41fafbd6b9d284fce06e" gracePeriod=45 Jul 09 02:38:02.369014 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.368955 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:38:02.467815 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.467789 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8534267e-58e2-459e-8332-1660b15ee85f-tls-certs\") pod \"8534267e-58e2-459e-8332-1660b15ee85f\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " Jul 09 02:38:02.468014 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.467830 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-model-cache\") pod \"8534267e-58e2-459e-8332-1660b15ee85f\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " Jul 09 02:38:02.468014 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.467864 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wv84b\" (UniqueName: \"kubernetes.io/projected/8534267e-58e2-459e-8332-1660b15ee85f-kube-api-access-wv84b\") pod \"8534267e-58e2-459e-8332-1660b15ee85f\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " Jul 09 02:38:02.468014 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.467985 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-tmp-dir\") pod \"8534267e-58e2-459e-8332-1660b15ee85f\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " Jul 09 02:38:02.468193 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.468036 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-dshm\") pod \"8534267e-58e2-459e-8332-1660b15ee85f\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " Jul 09 02:38:02.468193 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.468082 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-home\") pod \"8534267e-58e2-459e-8332-1660b15ee85f\" (UID: \"8534267e-58e2-459e-8332-1660b15ee85f\") " Jul 09 02:38:02.468193 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.468131 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-model-cache" (OuterVolumeSpecName: "model-cache") pod "8534267e-58e2-459e-8332-1660b15ee85f" (UID: "8534267e-58e2-459e-8332-1660b15ee85f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:38:02.468328 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.468252 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8534267e-58e2-459e-8332-1660b15ee85f" (UID: "8534267e-58e2-459e-8332-1660b15ee85f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:38:02.468376 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.468314 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-home" (OuterVolumeSpecName: "home") pod "8534267e-58e2-459e-8332-1660b15ee85f" (UID: "8534267e-58e2-459e-8332-1660b15ee85f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:38:02.468425 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.468375 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-model-cache\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:38:02.468425 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.468399 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-tmp-dir\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:38:02.470148 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.470126 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-dshm" (OuterVolumeSpecName: "dshm") pod "8534267e-58e2-459e-8332-1660b15ee85f" (UID: "8534267e-58e2-459e-8332-1660b15ee85f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:38:02.470409 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.470380 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8534267e-58e2-459e-8332-1660b15ee85f-kube-api-access-wv84b" (OuterVolumeSpecName: "kube-api-access-wv84b") pod "8534267e-58e2-459e-8332-1660b15ee85f" (UID: "8534267e-58e2-459e-8332-1660b15ee85f"). InnerVolumeSpecName "kube-api-access-wv84b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:38:02.470483 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.470423 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8534267e-58e2-459e-8332-1660b15ee85f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8534267e-58e2-459e-8332-1660b15ee85f" (UID: "8534267e-58e2-459e-8332-1660b15ee85f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 02:38:02.569285 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.569253 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-home\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:38:02.569285 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.569278 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8534267e-58e2-459e-8332-1660b15ee85f-tls-certs\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:38:02.569461 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.569292 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wv84b\" (UniqueName: \"kubernetes.io/projected/8534267e-58e2-459e-8332-1660b15ee85f-kube-api-access-wv84b\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:38:02.569461 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:02.569304 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8534267e-58e2-459e-8332-1660b15ee85f-dshm\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:38:03.042544 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:03.042514 2582 generic.go:356] "Generic (PLEG): container finished" podID="8534267e-58e2-459e-8332-1660b15ee85f" containerID="4cdae972e1ad8058ae59da03dbf2a03e797a28c8b8ac41fafbd6b9d284fce06e" exitCode=0 Jul 09 02:38:03.042739 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:03.042558 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" Jul 09 02:38:03.042739 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:03.042593 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" event={"ID":"8534267e-58e2-459e-8332-1660b15ee85f","Type":"ContainerDied","Data":"4cdae972e1ad8058ae59da03dbf2a03e797a28c8b8ac41fafbd6b9d284fce06e"} Jul 09 02:38:03.042739 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:03.042638 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk" event={"ID":"8534267e-58e2-459e-8332-1660b15ee85f","Type":"ContainerDied","Data":"d03920dec9691fa723bf6f2cd68169583eac65c81ad3f6c081ce05cbd8bd2aa8"} Jul 09 02:38:03.042739 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:03.042663 2582 scope.go:117] "RemoveContainer" containerID="4cdae972e1ad8058ae59da03dbf2a03e797a28c8b8ac41fafbd6b9d284fce06e" Jul 09 02:38:03.052358 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:03.052338 2582 scope.go:117] "RemoveContainer" containerID="4cdae972e1ad8058ae59da03dbf2a03e797a28c8b8ac41fafbd6b9d284fce06e" Jul 09 02:38:03.052612 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:38:03.052592 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cdae972e1ad8058ae59da03dbf2a03e797a28c8b8ac41fafbd6b9d284fce06e\": container with ID starting with 4cdae972e1ad8058ae59da03dbf2a03e797a28c8b8ac41fafbd6b9d284fce06e not found: ID does not exist" containerID="4cdae972e1ad8058ae59da03dbf2a03e797a28c8b8ac41fafbd6b9d284fce06e" Jul 09 02:38:03.052686 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:03.052622 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cdae972e1ad8058ae59da03dbf2a03e797a28c8b8ac41fafbd6b9d284fce06e"} err="failed to get container status \"4cdae972e1ad8058ae59da03dbf2a03e797a28c8b8ac41fafbd6b9d284fce06e\": rpc error: code = NotFound desc = could not find container \"4cdae972e1ad8058ae59da03dbf2a03e797a28c8b8ac41fafbd6b9d284fce06e\": container with ID starting with 4cdae972e1ad8058ae59da03dbf2a03e797a28c8b8ac41fafbd6b9d284fce06e not found: ID does not exist" Jul 09 02:38:03.064487 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:03.064460 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk"] Jul 09 02:38:03.069935 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:03.069912 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7124ed/sub-test-model-7124ed-kserve-6f759bdf57-rg8zk"] Jul 09 02:38:03.865828 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:03.865788 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8534267e-58e2-459e-8332-1660b15ee85f" path="/var/lib/kubelet/pods/8534267e-58e2-459e-8332-1660b15ee85f/volumes" Jul 09 02:38:10.109768 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:10.109732 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:38:14.391878 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:14.391838 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:38:52.721147 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:52.721085 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:38:54.981194 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:54.981159 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:38:55.080348 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:55.080313 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:38:56.690150 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:38:56.690114 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:39:00.658231 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:00.658187 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:39:00.757941 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:00.757865 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:39:08.741098 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:08.741052 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:39:08.838630 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:08.838591 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:39:12.176139 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:12.176105 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:39:27.587525 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:27.587491 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:39:28.190698 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:28.190659 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:39:34.723797 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:34.723754 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:39:35.608384 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:35.608351 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:39:36.537169 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:36.537131 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7362-140343328033712"] Jul 09 02:39:36.537553 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:36.537500 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8534267e-58e2-459e-8332-1660b15ee85f" containerName="main" Jul 09 02:39:36.537553 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:36.537511 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8534267e-58e2-459e-8332-1660b15ee85f" containerName="main" Jul 09 02:39:36.537626 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:36.537574 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8534267e-58e2-459e-8332-1660b15ee85f" containerName="main" Jul 09 02:39:36.540638 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:36.540617 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343328033712" Jul 09 02:39:36.543326 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:36.543306 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-7tnpn\"" Jul 09 02:39:36.547836 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:36.547655 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7362-140343328033712"] Jul 09 02:39:36.636778 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:36.636747 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgj7q\" (UniqueName: \"kubernetes.io/projected/02145206-225c-4fac-8fa5-e60e13d499fe-kube-api-access-rgj7q\") pod \"test-curl-7362-140343328033712\" (UID: \"02145206-225c-4fac-8fa5-e60e13d499fe\") " pod="openshift-ingress/test-curl-7362-140343328033712" Jul 09 02:39:36.738003 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:36.737963 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgj7q\" (UniqueName: \"kubernetes.io/projected/02145206-225c-4fac-8fa5-e60e13d499fe-kube-api-access-rgj7q\") pod \"test-curl-7362-140343328033712\" (UID: \"02145206-225c-4fac-8fa5-e60e13d499fe\") " pod="openshift-ingress/test-curl-7362-140343328033712" Jul 09 02:39:36.746221 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:36.746197 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgj7q\" (UniqueName: \"kubernetes.io/projected/02145206-225c-4fac-8fa5-e60e13d499fe-kube-api-access-rgj7q\") pod \"test-curl-7362-140343328033712\" (UID: \"02145206-225c-4fac-8fa5-e60e13d499fe\") " pod="openshift-ingress/test-curl-7362-140343328033712" Jul 09 02:39:36.850723 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:36.850651 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343328033712" Jul 09 02:39:36.980233 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:36.980209 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7362-140343328033712"] Jul 09 02:39:36.982161 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:39:36.982134 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02145206_225c_4fac_8fa5_e60e13d499fe.slice/crio-384487acc2e2a939cfe6ad8a1a94c8e85ecb30fa567ace53ddfc2834d7c337e8 WatchSource:0}: Error finding container 384487acc2e2a939cfe6ad8a1a94c8e85ecb30fa567ace53ddfc2834d7c337e8: Status 404 returned error can't find the container with id 384487acc2e2a939cfe6ad8a1a94c8e85ecb30fa567ace53ddfc2834d7c337e8 Jul 09 02:39:36.983803 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:36.983787 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 02:39:37.524620 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:37.524588 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7362-140343328033712" event={"ID":"02145206-225c-4fac-8fa5-e60e13d499fe","Type":"ContainerStarted","Data":"384487acc2e2a939cfe6ad8a1a94c8e85ecb30fa567ace53ddfc2834d7c337e8"} Jul 09 02:39:38.531283 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:38.531247 2582 generic.go:356] "Generic (PLEG): container finished" podID="02145206-225c-4fac-8fa5-e60e13d499fe" containerID="d9b2d647e7267875ba848f92410177cd3fd9d06c0ed51eb894da2c882c28f7a6" exitCode=0 Jul 09 02:39:38.531694 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:38.531343 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7362-140343328033712" event={"ID":"02145206-225c-4fac-8fa5-e60e13d499fe","Type":"ContainerDied","Data":"d9b2d647e7267875ba848f92410177cd3fd9d06c0ed51eb894da2c882c28f7a6"} Jul 09 02:39:39.679152 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:39.679130 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343328033712" Jul 09 02:39:39.701134 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:39.701098 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7362-140343328033712_02145206-225c-4fac-8fa5-e60e13d499fe/test-curl-7362-140343328033712/0.log" Jul 09 02:39:39.724704 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:39.724677 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7362-140343328033712"] Jul 09 02:39:39.730048 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:39.730024 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7362-140343328033712"] Jul 09 02:39:39.760122 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:39.760090 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgj7q\" (UniqueName: \"kubernetes.io/projected/02145206-225c-4fac-8fa5-e60e13d499fe-kube-api-access-rgj7q\") pod \"02145206-225c-4fac-8fa5-e60e13d499fe\" (UID: \"02145206-225c-4fac-8fa5-e60e13d499fe\") " Jul 09 02:39:39.762145 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:39.762115 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02145206-225c-4fac-8fa5-e60e13d499fe-kube-api-access-rgj7q" (OuterVolumeSpecName: "kube-api-access-rgj7q") pod "02145206-225c-4fac-8fa5-e60e13d499fe" (UID: "02145206-225c-4fac-8fa5-e60e13d499fe"). InnerVolumeSpecName "kube-api-access-rgj7q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:39:39.852419 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:39.852343 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7362-140343328034224"] Jul 09 02:39:39.852735 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:39.852721 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02145206-225c-4fac-8fa5-e60e13d499fe" containerName="test-curl-7362-140343328033712" Jul 09 02:39:39.852779 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:39.852737 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="02145206-225c-4fac-8fa5-e60e13d499fe" containerName="test-curl-7362-140343328033712" Jul 09 02:39:39.852820 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:39.852810 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="02145206-225c-4fac-8fa5-e60e13d499fe" containerName="test-curl-7362-140343328033712" Jul 09 02:39:39.855858 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:39.855840 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343328034224" Jul 09 02:39:39.861363 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:39.861341 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgj7q\" (UniqueName: \"kubernetes.io/projected/02145206-225c-4fac-8fa5-e60e13d499fe-kube-api-access-rgj7q\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:39:39.866560 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:39.866534 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02145206-225c-4fac-8fa5-e60e13d499fe" path="/var/lib/kubelet/pods/02145206-225c-4fac-8fa5-e60e13d499fe/volumes" Jul 09 02:39:39.866906 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:39.866892 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7362-140343328034224"] Jul 09 02:39:39.943810 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:39.943782 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:39:39.947851 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:39.947829 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:39:39.962733 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:39.962712 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx6cr\" (UniqueName: \"kubernetes.io/projected/0f90c50f-59fb-46c9-a339-f435f4c2973d-kube-api-access-tx6cr\") pod \"test-curl-7362-140343328034224\" (UID: \"0f90c50f-59fb-46c9-a339-f435f4c2973d\") " pod="openshift-ingress/test-curl-7362-140343328034224" Jul 09 02:39:40.063348 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:40.063313 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx6cr\" (UniqueName: \"kubernetes.io/projected/0f90c50f-59fb-46c9-a339-f435f4c2973d-kube-api-access-tx6cr\") pod \"test-curl-7362-140343328034224\" (UID: \"0f90c50f-59fb-46c9-a339-f435f4c2973d\") " pod="openshift-ingress/test-curl-7362-140343328034224" Jul 09 02:39:40.071560 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:40.071524 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx6cr\" (UniqueName: \"kubernetes.io/projected/0f90c50f-59fb-46c9-a339-f435f4c2973d-kube-api-access-tx6cr\") pod \"test-curl-7362-140343328034224\" (UID: \"0f90c50f-59fb-46c9-a339-f435f4c2973d\") " pod="openshift-ingress/test-curl-7362-140343328034224" Jul 09 02:39:40.165444 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:40.165414 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343328034224" Jul 09 02:39:40.292455 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:40.292432 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7362-140343328034224"] Jul 09 02:39:40.294130 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:39:40.294100 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f90c50f_59fb_46c9_a339_f435f4c2973d.slice/crio-1027331c7923e37a13e00388a8564712d21993784ed1b3b39101bb9f1004ca1f WatchSource:0}: Error finding container 1027331c7923e37a13e00388a8564712d21993784ed1b3b39101bb9f1004ca1f: Status 404 returned error can't find the container with id 1027331c7923e37a13e00388a8564712d21993784ed1b3b39101bb9f1004ca1f Jul 09 02:39:40.543188 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:40.543117 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343328033712" Jul 09 02:39:40.543188 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:40.543140 2582 scope.go:117] "RemoveContainer" containerID="d9b2d647e7267875ba848f92410177cd3fd9d06c0ed51eb894da2c882c28f7a6" Jul 09 02:39:40.544441 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:40.544414 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7362-140343328034224" event={"ID":"0f90c50f-59fb-46c9-a339-f435f4c2973d","Type":"ContainerStarted","Data":"1027331c7923e37a13e00388a8564712d21993784ed1b3b39101bb9f1004ca1f"} Jul 09 02:39:40.545843 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:39:40.545813 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7362-140343328033712\" is forbidden: User \"system:node:ip-10-0-130-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-184.ec2.internal' and this object" podUID="02145206-225c-4fac-8fa5-e60e13d499fe" pod="openshift-ingress/test-curl-7362-140343328033712" Jul 09 02:39:40.548938 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:39:40.548886 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7362-140343328033712\" is forbidden: User \"system:node:ip-10-0-130-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-184.ec2.internal' and this object" podUID="02145206-225c-4fac-8fa5-e60e13d499fe" pod="openshift-ingress/test-curl-7362-140343328033712" Jul 09 02:39:41.551209 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:41.551174 2582 generic.go:356] "Generic (PLEG): container finished" podID="0f90c50f-59fb-46c9-a339-f435f4c2973d" containerID="0a27059c440e2f7b346cb4dd8454f3f8b3367b01e4d5825f318e6c66ec38af2b" exitCode=0 Jul 09 02:39:41.551594 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:41.551262 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7362-140343328034224" event={"ID":"0f90c50f-59fb-46c9-a339-f435f4c2973d","Type":"ContainerDied","Data":"0a27059c440e2f7b346cb4dd8454f3f8b3367b01e4d5825f318e6c66ec38af2b"} Jul 09 02:39:41.553753 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:39:41.553732 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7362-140343328033712\" is forbidden: User \"system:node:ip-10-0-130-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-184.ec2.internal' and this object" podUID="02145206-225c-4fac-8fa5-e60e13d499fe" pod="openshift-ingress/test-curl-7362-140343328033712" Jul 09 02:39:41.686056 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:41.686026 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:39:42.696035 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:42.696009 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343328034224" Jul 09 02:39:42.715237 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:42.715215 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7362-140343328034224_0f90c50f-59fb-46c9-a339-f435f4c2973d/test-curl-7362-140343328034224/0.log" Jul 09 02:39:42.739445 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:42.739412 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7362-140343328034224"] Jul 09 02:39:42.741242 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:42.741215 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7362-140343328034224"] Jul 09 02:39:42.785081 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:42.785055 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tx6cr\" (UniqueName: \"kubernetes.io/projected/0f90c50f-59fb-46c9-a339-f435f4c2973d-kube-api-access-tx6cr\") pod \"0f90c50f-59fb-46c9-a339-f435f4c2973d\" (UID: \"0f90c50f-59fb-46c9-a339-f435f4c2973d\") " Jul 09 02:39:42.786928 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:42.786903 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f90c50f-59fb-46c9-a339-f435f4c2973d-kube-api-access-tx6cr" (OuterVolumeSpecName: "kube-api-access-tx6cr") pod "0f90c50f-59fb-46c9-a339-f435f4c2973d" (UID: "0f90c50f-59fb-46c9-a339-f435f4c2973d"). InnerVolumeSpecName "kube-api-access-tx6cr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:39:42.886308 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:42.886239 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tx6cr\" (UniqueName: \"kubernetes.io/projected/0f90c50f-59fb-46c9-a339-f435f4c2973d-kube-api-access-tx6cr\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:39:42.887111 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:42.887088 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7362-140343326873472"] Jul 09 02:39:42.887468 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:42.887452 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f90c50f-59fb-46c9-a339-f435f4c2973d" containerName="test-curl-7362-140343328034224" Jul 09 02:39:42.887536 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:42.887471 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f90c50f-59fb-46c9-a339-f435f4c2973d" containerName="test-curl-7362-140343328034224" Jul 09 02:39:42.887587 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:42.887572 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f90c50f-59fb-46c9-a339-f435f4c2973d" containerName="test-curl-7362-140343328034224" Jul 09 02:39:42.890613 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:42.890595 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343326873472" Jul 09 02:39:42.893759 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:42.893741 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7362-140343326873472"] Jul 09 02:39:42.987460 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:42.987433 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdx7q\" (UniqueName: \"kubernetes.io/projected/43afff7a-7570-4e01-9059-4e49ee4c8cad-kube-api-access-gdx7q\") pod \"test-curl-7362-140343326873472\" (UID: \"43afff7a-7570-4e01-9059-4e49ee4c8cad\") " pod="openshift-ingress/test-curl-7362-140343326873472" Jul 09 02:39:43.088307 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:43.088273 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdx7q\" (UniqueName: \"kubernetes.io/projected/43afff7a-7570-4e01-9059-4e49ee4c8cad-kube-api-access-gdx7q\") pod \"test-curl-7362-140343326873472\" (UID: \"43afff7a-7570-4e01-9059-4e49ee4c8cad\") " pod="openshift-ingress/test-curl-7362-140343326873472" Jul 09 02:39:43.096199 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:43.096171 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdx7q\" (UniqueName: \"kubernetes.io/projected/43afff7a-7570-4e01-9059-4e49ee4c8cad-kube-api-access-gdx7q\") pod \"test-curl-7362-140343326873472\" (UID: \"43afff7a-7570-4e01-9059-4e49ee4c8cad\") " pod="openshift-ingress/test-curl-7362-140343326873472" Jul 09 02:39:43.201696 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:43.201672 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343326873472" Jul 09 02:39:43.324084 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:43.324061 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7362-140343326873472"] Jul 09 02:39:43.325648 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:39:43.325619 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43afff7a_7570_4e01_9059_4e49ee4c8cad.slice/crio-f791e5b76b8758b558fd2d0c14c02f84706f01c5740c0990a48f9964222379a0 WatchSource:0}: Error finding container f791e5b76b8758b558fd2d0c14c02f84706f01c5740c0990a48f9964222379a0: Status 404 returned error can't find the container with id f791e5b76b8758b558fd2d0c14c02f84706f01c5740c0990a48f9964222379a0 Jul 09 02:39:43.563302 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:43.563205 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7362-140343326873472" event={"ID":"43afff7a-7570-4e01-9059-4e49ee4c8cad","Type":"ContainerStarted","Data":"f791e5b76b8758b558fd2d0c14c02f84706f01c5740c0990a48f9964222379a0"} Jul 09 02:39:43.564518 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:43.564498 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1027331c7923e37a13e00388a8564712d21993784ed1b3b39101bb9f1004ca1f" Jul 09 02:39:43.564608 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:43.564533 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343328034224" Jul 09 02:39:43.574894 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:39:43.574853 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7362-140343328034224\" is forbidden: User \"system:node:ip-10-0-130-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-184.ec2.internal' and this object" podUID="0f90c50f-59fb-46c9-a339-f435f4c2973d" pod="openshift-ingress/test-curl-7362-140343328034224" Jul 09 02:39:43.866119 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:43.866054 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f90c50f-59fb-46c9-a339-f435f4c2973d" path="/var/lib/kubelet/pods/0f90c50f-59fb-46c9-a339-f435f4c2973d/volumes" Jul 09 02:39:44.571746 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:44.571713 2582 generic.go:356] "Generic (PLEG): container finished" podID="43afff7a-7570-4e01-9059-4e49ee4c8cad" containerID="e28e295d08b1cf6e91e7501146bc9e24a0992c66f1be78e415c8a227c6239506" exitCode=0 Jul 09 02:39:44.571921 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:44.571756 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7362-140343326873472" event={"ID":"43afff7a-7570-4e01-9059-4e49ee4c8cad","Type":"ContainerDied","Data":"e28e295d08b1cf6e91e7501146bc9e24a0992c66f1be78e415c8a227c6239506"} Jul 09 02:39:45.724313 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:45.724290 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343326873472" Jul 09 02:39:45.746232 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:45.746186 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7362-140343326873472_43afff7a-7570-4e01-9059-4e49ee4c8cad/test-curl-7362-140343326873472/0.log" Jul 09 02:39:45.772052 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:45.772028 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7362-140343326873472"] Jul 09 02:39:45.773912 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:45.773893 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7362-140343326873472"] Jul 09 02:39:45.811539 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:45.811512 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdx7q\" (UniqueName: \"kubernetes.io/projected/43afff7a-7570-4e01-9059-4e49ee4c8cad-kube-api-access-gdx7q\") pod \"43afff7a-7570-4e01-9059-4e49ee4c8cad\" (UID: \"43afff7a-7570-4e01-9059-4e49ee4c8cad\") " Jul 09 02:39:45.813476 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:45.813449 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43afff7a-7570-4e01-9059-4e49ee4c8cad-kube-api-access-gdx7q" (OuterVolumeSpecName: "kube-api-access-gdx7q") pod "43afff7a-7570-4e01-9059-4e49ee4c8cad" (UID: "43afff7a-7570-4e01-9059-4e49ee4c8cad"). InnerVolumeSpecName "kube-api-access-gdx7q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:39:45.865690 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:45.865627 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43afff7a-7570-4e01-9059-4e49ee4c8cad" path="/var/lib/kubelet/pods/43afff7a-7570-4e01-9059-4e49ee4c8cad/volumes" Jul 09 02:39:45.886827 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:45.886800 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7362-140343326873472"] Jul 09 02:39:45.887162 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:45.887147 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43afff7a-7570-4e01-9059-4e49ee4c8cad" containerName="test-curl-7362-140343326873472" Jul 09 02:39:45.887208 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:45.887164 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="43afff7a-7570-4e01-9059-4e49ee4c8cad" containerName="test-curl-7362-140343326873472" Jul 09 02:39:45.887240 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:45.887231 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="43afff7a-7570-4e01-9059-4e49ee4c8cad" containerName="test-curl-7362-140343326873472" Jul 09 02:39:45.890088 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:45.890073 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343326873472" Jul 09 02:39:45.895814 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:45.895792 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7362-140343326873472"] Jul 09 02:39:45.912682 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:45.912654 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7znc\" (UniqueName: \"kubernetes.io/projected/fb472ded-9d7e-4634-91b9-7c55e174c68f-kube-api-access-v7znc\") pod \"test-curl-7362-140343326873472\" (UID: \"fb472ded-9d7e-4634-91b9-7c55e174c68f\") " pod="openshift-ingress/test-curl-7362-140343326873472" Jul 09 02:39:45.912799 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:45.912746 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdx7q\" (UniqueName: \"kubernetes.io/projected/43afff7a-7570-4e01-9059-4e49ee4c8cad-kube-api-access-gdx7q\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:39:46.013811 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:46.013776 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7znc\" (UniqueName: \"kubernetes.io/projected/fb472ded-9d7e-4634-91b9-7c55e174c68f-kube-api-access-v7znc\") pod \"test-curl-7362-140343326873472\" (UID: \"fb472ded-9d7e-4634-91b9-7c55e174c68f\") " pod="openshift-ingress/test-curl-7362-140343326873472" Jul 09 02:39:46.021882 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:46.021841 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7znc\" (UniqueName: \"kubernetes.io/projected/fb472ded-9d7e-4634-91b9-7c55e174c68f-kube-api-access-v7znc\") pod \"test-curl-7362-140343326873472\" (UID: \"fb472ded-9d7e-4634-91b9-7c55e174c68f\") " pod="openshift-ingress/test-curl-7362-140343326873472" Jul 09 02:39:46.199506 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:46.199475 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343326873472" Jul 09 02:39:46.329160 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:46.329132 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7362-140343326873472"] Jul 09 02:39:46.330787 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:39:46.330756 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb472ded_9d7e_4634_91b9_7c55e174c68f.slice/crio-55e21fd0250c6c657fc7f64aca25fabda052f8d8a049026023dd461fe4d1965c WatchSource:0}: Error finding container 55e21fd0250c6c657fc7f64aca25fabda052f8d8a049026023dd461fe4d1965c: Status 404 returned error can't find the container with id 55e21fd0250c6c657fc7f64aca25fabda052f8d8a049026023dd461fe4d1965c Jul 09 02:39:46.583530 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:46.583437 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343326873472" Jul 09 02:39:46.583530 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:46.583494 2582 scope.go:117] "RemoveContainer" containerID="e28e295d08b1cf6e91e7501146bc9e24a0992c66f1be78e415c8a227c6239506" Jul 09 02:39:46.584778 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:46.584676 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7362-140343326873472" event={"ID":"fb472ded-9d7e-4634-91b9-7c55e174c68f","Type":"ContainerStarted","Data":"55e21fd0250c6c657fc7f64aca25fabda052f8d8a049026023dd461fe4d1965c"} Jul 09 02:39:47.592281 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:47.592241 2582 generic.go:356] "Generic (PLEG): container finished" podID="fb472ded-9d7e-4634-91b9-7c55e174c68f" containerID="adb9fe35f61d9b72330091a034b45a8f001e9f44fe8efea077e75e0e8d559957" exitCode=0 Jul 09 02:39:47.592657 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:47.592327 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7362-140343326873472" event={"ID":"fb472ded-9d7e-4634-91b9-7c55e174c68f","Type":"ContainerDied","Data":"adb9fe35f61d9b72330091a034b45a8f001e9f44fe8efea077e75e0e8d559957"} Jul 09 02:39:48.746585 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:48.746565 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343326873472" Jul 09 02:39:48.765406 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:48.765383 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7362-140343326873472_fb472ded-9d7e-4634-91b9-7c55e174c68f/test-curl-7362-140343326873472/0.log" Jul 09 02:39:48.785448 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:48.785422 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7362-140343326873472"] Jul 09 02:39:48.787431 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:48.787411 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7362-140343326873472"] Jul 09 02:39:48.835312 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:48.835287 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7znc\" (UniqueName: \"kubernetes.io/projected/fb472ded-9d7e-4634-91b9-7c55e174c68f-kube-api-access-v7znc\") pod \"fb472ded-9d7e-4634-91b9-7c55e174c68f\" (UID: \"fb472ded-9d7e-4634-91b9-7c55e174c68f\") " Jul 09 02:39:48.837382 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:48.837358 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb472ded-9d7e-4634-91b9-7c55e174c68f-kube-api-access-v7znc" (OuterVolumeSpecName: "kube-api-access-v7znc") pod "fb472ded-9d7e-4634-91b9-7c55e174c68f" (UID: "fb472ded-9d7e-4634-91b9-7c55e174c68f"). InnerVolumeSpecName "kube-api-access-v7znc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:39:48.898075 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:48.898050 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7362-140343326874480"] Jul 09 02:39:48.898416 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:48.898402 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb472ded-9d7e-4634-91b9-7c55e174c68f" containerName="test-curl-7362-140343326873472" Jul 09 02:39:48.898470 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:48.898418 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb472ded-9d7e-4634-91b9-7c55e174c68f" containerName="test-curl-7362-140343326873472" Jul 09 02:39:48.898506 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:48.898479 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb472ded-9d7e-4634-91b9-7c55e174c68f" containerName="test-curl-7362-140343326873472" Jul 09 02:39:48.902177 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:48.902131 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343326874480" Jul 09 02:39:48.907504 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:48.907479 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7362-140343326874480"] Jul 09 02:39:48.936704 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:48.936671 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc5zx\" (UniqueName: \"kubernetes.io/projected/e6744988-57bb-4000-832b-9eb7e440845a-kube-api-access-hc5zx\") pod \"test-curl-7362-140343326874480\" (UID: \"e6744988-57bb-4000-832b-9eb7e440845a\") " pod="openshift-ingress/test-curl-7362-140343326874480" Jul 09 02:39:48.936854 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:48.936817 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7znc\" (UniqueName: \"kubernetes.io/projected/fb472ded-9d7e-4634-91b9-7c55e174c68f-kube-api-access-v7znc\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:39:49.038040 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:49.038007 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc5zx\" (UniqueName: \"kubernetes.io/projected/e6744988-57bb-4000-832b-9eb7e440845a-kube-api-access-hc5zx\") pod \"test-curl-7362-140343326874480\" (UID: \"e6744988-57bb-4000-832b-9eb7e440845a\") " pod="openshift-ingress/test-curl-7362-140343326874480" Jul 09 02:39:49.046403 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:49.046373 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc5zx\" (UniqueName: \"kubernetes.io/projected/e6744988-57bb-4000-832b-9eb7e440845a-kube-api-access-hc5zx\") pod \"test-curl-7362-140343326874480\" (UID: \"e6744988-57bb-4000-832b-9eb7e440845a\") " pod="openshift-ingress/test-curl-7362-140343326874480" Jul 09 02:39:49.214414 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:49.214340 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343326874480" Jul 09 02:39:49.343701 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:49.343672 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7362-140343326874480"] Jul 09 02:39:49.345242 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:39:49.345212 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6744988_57bb_4000_832b_9eb7e440845a.slice/crio-024075475baeaa9fa3d53dd2fbc9171dde42f00487889a20ab75ed3fd8de284e WatchSource:0}: Error finding container 024075475baeaa9fa3d53dd2fbc9171dde42f00487889a20ab75ed3fd8de284e: Status 404 returned error can't find the container with id 024075475baeaa9fa3d53dd2fbc9171dde42f00487889a20ab75ed3fd8de284e Jul 09 02:39:49.603729 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:49.603647 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7362-140343326874480" event={"ID":"e6744988-57bb-4000-832b-9eb7e440845a","Type":"ContainerStarted","Data":"024075475baeaa9fa3d53dd2fbc9171dde42f00487889a20ab75ed3fd8de284e"} Jul 09 02:39:49.604865 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:49.604845 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343326873472" Jul 09 02:39:49.605007 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:49.604889 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55e21fd0250c6c657fc7f64aca25fabda052f8d8a049026023dd461fe4d1965c" Jul 09 02:39:49.615486 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:39:49.615461 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7362-140343326873472\" is forbidden: User \"system:node:ip-10-0-130-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-184.ec2.internal' and this object" podUID="fb472ded-9d7e-4634-91b9-7c55e174c68f" pod="openshift-ingress/test-curl-7362-140343326873472" Jul 09 02:39:49.865137 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:39:49.865057 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7362-140343326873472\" is forbidden: User \"system:node:ip-10-0-130-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-184.ec2.internal' and this object" podUID="fb472ded-9d7e-4634-91b9-7c55e174c68f" pod="openshift-ingress/test-curl-7362-140343326873472" Jul 09 02:39:49.867022 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:49.867001 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb472ded-9d7e-4634-91b9-7c55e174c68f" path="/var/lib/kubelet/pods/fb472ded-9d7e-4634-91b9-7c55e174c68f/volumes" Jul 09 02:39:50.611276 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:50.611245 2582 generic.go:356] "Generic (PLEG): container finished" podID="e6744988-57bb-4000-832b-9eb7e440845a" containerID="3733a7859dd2268e5490140582d169bfff580167457632265f177cdc4a3f2843" exitCode=0 Jul 09 02:39:50.611488 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:50.611327 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7362-140343326874480" event={"ID":"e6744988-57bb-4000-832b-9eb7e440845a","Type":"ContainerDied","Data":"3733a7859dd2268e5490140582d169bfff580167457632265f177cdc4a3f2843"} Jul 09 02:39:51.753891 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:51.753855 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343326874480" Jul 09 02:39:51.775914 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:51.775865 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7362-140343326874480_e6744988-57bb-4000-832b-9eb7e440845a/test-curl-7362-140343326874480/0.log" Jul 09 02:39:51.797078 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:51.797050 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7362-140343326874480"] Jul 09 02:39:51.800254 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:51.800233 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7362-140343326874480"] Jul 09 02:39:51.860015 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:51.859993 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hc5zx\" (UniqueName: \"kubernetes.io/projected/e6744988-57bb-4000-832b-9eb7e440845a-kube-api-access-hc5zx\") pod \"e6744988-57bb-4000-832b-9eb7e440845a\" (UID: \"e6744988-57bb-4000-832b-9eb7e440845a\") " Jul 09 02:39:51.862029 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:51.862002 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6744988-57bb-4000-832b-9eb7e440845a-kube-api-access-hc5zx" (OuterVolumeSpecName: "kube-api-access-hc5zx") pod "e6744988-57bb-4000-832b-9eb7e440845a" (UID: "e6744988-57bb-4000-832b-9eb7e440845a"). InnerVolumeSpecName "kube-api-access-hc5zx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:39:51.865642 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:51.865620 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6744988-57bb-4000-832b-9eb7e440845a" path="/var/lib/kubelet/pods/e6744988-57bb-4000-832b-9eb7e440845a/volumes" Jul 09 02:39:51.961364 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:51.961344 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hc5zx\" (UniqueName: \"kubernetes.io/projected/e6744988-57bb-4000-832b-9eb7e440845a-kube-api-access-hc5zx\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:39:52.621667 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:52.621639 2582 scope.go:117] "RemoveContainer" containerID="3733a7859dd2268e5490140582d169bfff580167457632265f177cdc4a3f2843" Jul 09 02:39:52.621667 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:52.621667 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7362-140343326874480" Jul 09 02:39:55.136428 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:55.136395 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:39:55.525746 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:55.525717 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:39:55.626345 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:55.626313 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:39:55.753222 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:55.753181 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:39:59.487477 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:39:59.487447 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:40:01.684648 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:01.684554 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:40:01.785536 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:01.785508 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:40:01.933698 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:01.933669 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:40:02.154021 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:02.153984 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:40:02.299416 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:02.299382 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:40:02.694480 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:02.694447 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:40:02.793719 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:02.793677 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:40:02.917169 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:02.917141 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:40:08.772009 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:08.771977 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:40:08.829278 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:08.829246 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:40:09.017295 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:09.017264 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr"] Jul 09 02:40:09.017520 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:09.017496 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" podUID="935cd0ff-9d88-4db0-9856-fd50f784a542" containerName="istio-proxy" containerID="cri-o://f4dba92fc804cf9c2f2990732bd69fb1ebc6c969c2f4dafa54751f2c977fff90" gracePeriod=30 Jul 09 02:40:09.031950 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:09.031805 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:40:09.250086 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:09.250052 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:40:11.584364 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:11.584331 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r7rtl"] Jul 09 02:40:14.289276 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.289219 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:40:14.326183 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.326149 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-rkl5n_bf4f6c90-4abd-4907-b6cb-58b8f67af948/manager/0.log" Jul 09 02:40:14.447209 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.447177 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-dnrfl_433ba04e-75c0-4ce1-8793-aec0e8ff93d6/manager/0.log" Jul 09 02:40:14.457105 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.457077 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-workload-certs\") pod \"935cd0ff-9d88-4db0-9856-fd50f784a542\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " Jul 09 02:40:14.457230 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.457144 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjdvw\" (UniqueName: \"kubernetes.io/projected/935cd0ff-9d88-4db0-9856-fd50f784a542-kube-api-access-fjdvw\") pod \"935cd0ff-9d88-4db0-9856-fd50f784a542\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " Jul 09 02:40:14.457230 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.457173 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-credential-socket\") pod \"935cd0ff-9d88-4db0-9856-fd50f784a542\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " Jul 09 02:40:14.457230 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.457198 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-workload-socket\") pod \"935cd0ff-9d88-4db0-9856-fd50f784a542\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " Jul 09 02:40:14.457401 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.457251 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-podinfo\") pod \"935cd0ff-9d88-4db0-9856-fd50f784a542\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " Jul 09 02:40:14.457401 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.457280 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-token\") pod \"935cd0ff-9d88-4db0-9856-fd50f784a542\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " Jul 09 02:40:14.457401 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.457306 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-data\") pod \"935cd0ff-9d88-4db0-9856-fd50f784a542\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " Jul 09 02:40:14.457401 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.457346 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/935cd0ff-9d88-4db0-9856-fd50f784a542-istiod-ca-cert\") pod \"935cd0ff-9d88-4db0-9856-fd50f784a542\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " Jul 09 02:40:14.457401 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.457387 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-envoy\") pod \"935cd0ff-9d88-4db0-9856-fd50f784a542\" (UID: \"935cd0ff-9d88-4db0-9856-fd50f784a542\") " Jul 09 02:40:14.457653 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.457371 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "935cd0ff-9d88-4db0-9856-fd50f784a542" (UID: "935cd0ff-9d88-4db0-9856-fd50f784a542"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:40:14.457653 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.457469 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "935cd0ff-9d88-4db0-9856-fd50f784a542" (UID: "935cd0ff-9d88-4db0-9856-fd50f784a542"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:40:14.457653 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.457538 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "935cd0ff-9d88-4db0-9856-fd50f784a542" (UID: "935cd0ff-9d88-4db0-9856-fd50f784a542"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:40:14.457821 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.457676 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-workload-certs\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:40:14.457821 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.457694 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-credential-socket\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:40:14.457821 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.457708 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-workload-socket\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:40:14.457821 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.457765 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/935cd0ff-9d88-4db0-9856-fd50f784a542-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "935cd0ff-9d88-4db0-9856-fd50f784a542" (UID: "935cd0ff-9d88-4db0-9856-fd50f784a542"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 02:40:14.458198 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.458171 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-data" (OuterVolumeSpecName: "istio-data") pod "935cd0ff-9d88-4db0-9856-fd50f784a542" (UID: "935cd0ff-9d88-4db0-9856-fd50f784a542"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:40:14.459399 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.459372 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/935cd0ff-9d88-4db0-9856-fd50f784a542-kube-api-access-fjdvw" (OuterVolumeSpecName: "kube-api-access-fjdvw") pod "935cd0ff-9d88-4db0-9856-fd50f784a542" (UID: "935cd0ff-9d88-4db0-9856-fd50f784a542"). InnerVolumeSpecName "kube-api-access-fjdvw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:40:14.459516 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.459465 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-token" (OuterVolumeSpecName: "istio-token") pod "935cd0ff-9d88-4db0-9856-fd50f784a542" (UID: "935cd0ff-9d88-4db0-9856-fd50f784a542"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 02:40:14.459744 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.459723 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "935cd0ff-9d88-4db0-9856-fd50f784a542" (UID: "935cd0ff-9d88-4db0-9856-fd50f784a542"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 02:40:14.459803 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.459724 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "935cd0ff-9d88-4db0-9856-fd50f784a542" (UID: "935cd0ff-9d88-4db0-9856-fd50f784a542"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 02:40:14.558215 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.558135 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjdvw\" (UniqueName: \"kubernetes.io/projected/935cd0ff-9d88-4db0-9856-fd50f784a542-kube-api-access-fjdvw\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:40:14.558215 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.558166 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-podinfo\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:40:14.558215 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.558177 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-token\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:40:14.558215 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.558186 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-data\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:40:14.558215 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.558195 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/935cd0ff-9d88-4db0-9856-fd50f784a542-istiod-ca-cert\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:40:14.558215 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.558207 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/935cd0ff-9d88-4db0-9856-fd50f784a542-istio-envoy\") on node \"ip-10-0-130-184.ec2.internal\" DevicePath \"\"" Jul 09 02:40:14.564556 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.564533 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-669bbdcddd-9cp7q_27cf0c1a-9674-4132-b220-9bf4dc6834c1/manager/0.log" Jul 09 02:40:14.681740 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.681710 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-8cstw_fc976687-df22-4397-a0dd-797c604bc7a6/server/0.log" Jul 09 02:40:14.737167 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.737131 2582 generic.go:356] "Generic (PLEG): container finished" podID="935cd0ff-9d88-4db0-9856-fd50f784a542" containerID="f4dba92fc804cf9c2f2990732bd69fb1ebc6c969c2f4dafa54751f2c977fff90" exitCode=0 Jul 09 02:40:14.737336 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.737183 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" Jul 09 02:40:14.737336 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.737212 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" event={"ID":"935cd0ff-9d88-4db0-9856-fd50f784a542","Type":"ContainerDied","Data":"f4dba92fc804cf9c2f2990732bd69fb1ebc6c969c2f4dafa54751f2c977fff90"} Jul 09 02:40:14.737336 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.737291 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr" event={"ID":"935cd0ff-9d88-4db0-9856-fd50f784a542","Type":"ContainerDied","Data":"99c533008328f174ac3ae48b113274d63d57ed7629d8fe5a8cc6adf208077b39"} Jul 09 02:40:14.737336 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.737315 2582 scope.go:117] "RemoveContainer" containerID="f4dba92fc804cf9c2f2990732bd69fb1ebc6c969c2f4dafa54751f2c977fff90" Jul 09 02:40:14.747583 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.747566 2582 scope.go:117] "RemoveContainer" containerID="04b8ba949996b1f6e05f2da3b8930331bd48b0af35ee8df25c66691128bc01bc" Jul 09 02:40:14.756090 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.756074 2582 scope.go:117] "RemoveContainer" containerID="f4dba92fc804cf9c2f2990732bd69fb1ebc6c969c2f4dafa54751f2c977fff90" Jul 09 02:40:14.756341 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:40:14.756322 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4dba92fc804cf9c2f2990732bd69fb1ebc6c969c2f4dafa54751f2c977fff90\": container with ID starting with f4dba92fc804cf9c2f2990732bd69fb1ebc6c969c2f4dafa54751f2c977fff90 not found: ID does not exist" containerID="f4dba92fc804cf9c2f2990732bd69fb1ebc6c969c2f4dafa54751f2c977fff90" Jul 09 02:40:14.756388 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.756348 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4dba92fc804cf9c2f2990732bd69fb1ebc6c969c2f4dafa54751f2c977fff90"} err="failed to get container status \"f4dba92fc804cf9c2f2990732bd69fb1ebc6c969c2f4dafa54751f2c977fff90\": rpc error: code = NotFound desc = could not find container \"f4dba92fc804cf9c2f2990732bd69fb1ebc6c969c2f4dafa54751f2c977fff90\": container with ID starting with f4dba92fc804cf9c2f2990732bd69fb1ebc6c969c2f4dafa54751f2c977fff90 not found: ID does not exist" Jul 09 02:40:14.756388 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.756364 2582 scope.go:117] "RemoveContainer" containerID="04b8ba949996b1f6e05f2da3b8930331bd48b0af35ee8df25c66691128bc01bc" Jul 09 02:40:14.756651 ip-10-0-130-184 kubenswrapper[2582]: E0709 02:40:14.756631 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04b8ba949996b1f6e05f2da3b8930331bd48b0af35ee8df25c66691128bc01bc\": container with ID starting with 04b8ba949996b1f6e05f2da3b8930331bd48b0af35ee8df25c66691128bc01bc not found: ID does not exist" containerID="04b8ba949996b1f6e05f2da3b8930331bd48b0af35ee8df25c66691128bc01bc" Jul 09 02:40:14.756704 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.756656 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04b8ba949996b1f6e05f2da3b8930331bd48b0af35ee8df25c66691128bc01bc"} err="failed to get container status \"04b8ba949996b1f6e05f2da3b8930331bd48b0af35ee8df25c66691128bc01bc\": rpc error: code = NotFound desc = could not find container \"04b8ba949996b1f6e05f2da3b8930331bd48b0af35ee8df25c66691128bc01bc\": container with ID starting with 04b8ba949996b1f6e05f2da3b8930331bd48b0af35ee8df25c66691128bc01bc not found: ID does not exist" Jul 09 02:40:14.760966 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.760944 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr"] Jul 09 02:40:14.763992 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.763971 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-c6afad-openshift-default-5484fd4f59-r9jwr"] Jul 09 02:40:14.798435 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:14.798417 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2d6sg_06e1f2d4-cf8d-4d3b-8123-4741b828125f/manager/0.log" Jul 09 02:40:15.181681 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:15.181656 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8687897cbb-gbnt5_2c5cb7ea-589c-41ab-a5ca-54ada4284c7d/copy-manifests/0.log" Jul 09 02:40:15.201407 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:15.201364 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8687897cbb-gbnt5_2c5cb7ea-589c-41ab-a5ca-54ada4284c7d/manager/0.log" Jul 09 02:40:15.866188 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:15.866156 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="935cd0ff-9d88-4db0-9856-fd50f784a542" path="/var/lib/kubelet/pods/935cd0ff-9d88-4db0-9856-fd50f784a542/volumes" Jul 09 02:40:16.418582 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:16.418550 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l_34076dc0-01aa-4322-b5a4-ccf17bb46fad/pull/0.log" Jul 09 02:40:16.423954 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:16.423928 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l_34076dc0-01aa-4322-b5a4-ccf17bb46fad/extract/0.log" Jul 09 02:40:16.428821 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:16.428801 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l_34076dc0-01aa-4322-b5a4-ccf17bb46fad/util/0.log" Jul 09 02:40:16.535149 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:16.535124 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2_b19afe3c-311b-4c34-94ef-975696e39caf/pull/0.log" Jul 09 02:40:16.540630 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:16.540610 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2_b19afe3c-311b-4c34-94ef-975696e39caf/extract/0.log" Jul 09 02:40:16.545631 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:16.545614 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2_b19afe3c-311b-4c34-94ef-975696e39caf/util/0.log" Jul 09 02:40:16.658701 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:16.658680 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh_261f21fc-52be-4e8c-82ca-807f6ca6513d/pull/0.log" Jul 09 02:40:16.665080 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:16.665065 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh_261f21fc-52be-4e8c-82ca-807f6ca6513d/extract/0.log" Jul 09 02:40:16.670766 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:16.670710 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh_261f21fc-52be-4e8c-82ca-807f6ca6513d/util/0.log" Jul 09 02:40:16.779995 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:16.779971 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf_b859e0af-2e73-42e4-a3f4-36cc346aac92/util/0.log" Jul 09 02:40:16.785430 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:16.785409 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf_b859e0af-2e73-42e4-a3f4-36cc346aac92/pull/0.log" Jul 09 02:40:16.790847 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:16.790822 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf_b859e0af-2e73-42e4-a3f4-36cc346aac92/extract/0.log" Jul 09 02:40:17.018117 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:17.018043 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-x5tmh_eb5a1a9f-9cd5-4ebf-a070-ed56eec53f18/manager/0.log" Jul 09 02:40:17.123748 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:17.123715 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-lngv4_ca1cefe0-a5f8-4793-a7c6-ff21c7996095/manager/0.log" Jul 09 02:40:17.351860 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:17.351776 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-52mq2_9137fc0a-da4d-4f44-a462-32a6a81e8571/registry-server/0.log" Jul 09 02:40:17.476669 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:17.476642 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-rsbcx_c7f67406-2d81-48fa-8c0f-d3855f65eddf/manager/0.log" Jul 09 02:40:17.592509 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:17.592483 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-r7rtl_300bc103-5b60-4d66-9418-885fa3e554bb/limitador/0.log" Jul 09 02:40:17.716608 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:17.716576 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-q8zh5_8f5752c8-a347-415c-9cb4-5224c81bd875/manager/0.log" Jul 09 02:40:18.282860 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:18.282828 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-847fc5468-8lr4q_e3f31f39-5c9a-4fda-86fa-331f25d1f555/kube-auth-proxy/0.log" Jul 09 02:40:19.508840 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:19.508810 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtxm6_0d246772-b923-4799-8af0-50af7a16d280/main/0.log" Jul 09 02:40:19.734860 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:19.734827 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-znqr8_7f345d6e-1b9b-440e-a83e-07df6109368b/main/0.log" Jul 09 02:40:23.272754 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.272718 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lshzb/must-gather-xq6c8"] Jul 09 02:40:23.273147 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.273063 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="935cd0ff-9d88-4db0-9856-fd50f784a542" containerName="istio-proxy" Jul 09 02:40:23.273147 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.273078 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="935cd0ff-9d88-4db0-9856-fd50f784a542" containerName="istio-proxy" Jul 09 02:40:23.273147 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.273106 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6744988-57bb-4000-832b-9eb7e440845a" containerName="test-curl-7362-140343326874480" Jul 09 02:40:23.273147 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.273119 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6744988-57bb-4000-832b-9eb7e440845a" containerName="test-curl-7362-140343326874480" Jul 09 02:40:23.273328 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.273227 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6744988-57bb-4000-832b-9eb7e440845a" containerName="test-curl-7362-140343326874480" Jul 09 02:40:23.273328 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.273243 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="935cd0ff-9d88-4db0-9856-fd50f784a542" containerName="istio-proxy" Jul 09 02:40:23.273328 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.273254 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="935cd0ff-9d88-4db0-9856-fd50f784a542" containerName="istio-proxy" Jul 09 02:40:23.273486 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.273379 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="935cd0ff-9d88-4db0-9856-fd50f784a542" containerName="istio-proxy" Jul 09 02:40:23.273486 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.273388 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="935cd0ff-9d88-4db0-9856-fd50f784a542" containerName="istio-proxy" Jul 09 02:40:23.277522 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.277507 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lshzb/must-gather-xq6c8" Jul 09 02:40:23.280409 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.280391 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lshzb\"/\"openshift-service-ca.crt\"" Jul 09 02:40:23.281829 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.281812 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lshzb\"/\"default-dockercfg-mbz6n\"" Jul 09 02:40:23.281949 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.281891 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lshzb\"/\"kube-root-ca.crt\"" Jul 09 02:40:23.285613 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.285590 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lshzb/must-gather-xq6c8"] Jul 09 02:40:23.286614 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.286591 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:40:23.379612 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.379588 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" limit=1024 Jul 09 02:40:23.428527 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.428504 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfwbz\" (UniqueName: \"kubernetes.io/projected/384a9156-93b8-433b-a6f1-c9a9899012d1-kube-api-access-mfwbz\") pod \"must-gather-xq6c8\" (UID: \"384a9156-93b8-433b-a6f1-c9a9899012d1\") " pod="openshift-must-gather-lshzb/must-gather-xq6c8" Jul 09 02:40:23.428619 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.428553 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/384a9156-93b8-433b-a6f1-c9a9899012d1-must-gather-output\") pod \"must-gather-xq6c8\" (UID: \"384a9156-93b8-433b-a6f1-c9a9899012d1\") " pod="openshift-must-gather-lshzb/must-gather-xq6c8" Jul 09 02:40:23.529659 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.529602 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfwbz\" (UniqueName: \"kubernetes.io/projected/384a9156-93b8-433b-a6f1-c9a9899012d1-kube-api-access-mfwbz\") pod \"must-gather-xq6c8\" (UID: \"384a9156-93b8-433b-a6f1-c9a9899012d1\") " pod="openshift-must-gather-lshzb/must-gather-xq6c8" Jul 09 02:40:23.529659 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.529647 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/384a9156-93b8-433b-a6f1-c9a9899012d1-must-gather-output\") pod \"must-gather-xq6c8\" (UID: \"384a9156-93b8-433b-a6f1-c9a9899012d1\") " pod="openshift-must-gather-lshzb/must-gather-xq6c8" Jul 09 02:40:23.529942 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.529927 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/384a9156-93b8-433b-a6f1-c9a9899012d1-must-gather-output\") pod \"must-gather-xq6c8\" (UID: \"384a9156-93b8-433b-a6f1-c9a9899012d1\") " pod="openshift-must-gather-lshzb/must-gather-xq6c8" Jul 09 02:40:23.538165 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.538143 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfwbz\" (UniqueName: \"kubernetes.io/projected/384a9156-93b8-433b-a6f1-c9a9899012d1-kube-api-access-mfwbz\") pod \"must-gather-xq6c8\" (UID: \"384a9156-93b8-433b-a6f1-c9a9899012d1\") " pod="openshift-must-gather-lshzb/must-gather-xq6c8" Jul 09 02:40:23.588076 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.588054 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lshzb/must-gather-xq6c8" Jul 09 02:40:23.715006 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.714979 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lshzb/must-gather-xq6c8"] Jul 09 02:40:23.716459 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:40:23.716424 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod384a9156_93b8_433b_a6f1_c9a9899012d1.slice/crio-4fbf4e5bdfba4e758ce7c9ac6e318f74b96380d257e1a134f43cddf4912924f3 WatchSource:0}: Error finding container 4fbf4e5bdfba4e758ce7c9ac6e318f74b96380d257e1a134f43cddf4912924f3: Status 404 returned error can't find the container with id 4fbf4e5bdfba4e758ce7c9ac6e318f74b96380d257e1a134f43cddf4912924f3 Jul 09 02:40:23.790887 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:23.785455 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lshzb/must-gather-xq6c8" event={"ID":"384a9156-93b8-433b-a6f1-c9a9899012d1","Type":"ContainerStarted","Data":"4fbf4e5bdfba4e758ce7c9ac6e318f74b96380d257e1a134f43cddf4912924f3"} Jul 09 02:40:25.800507 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:25.800470 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lshzb/must-gather-xq6c8" event={"ID":"384a9156-93b8-433b-a6f1-c9a9899012d1","Type":"ContainerStarted","Data":"6b844fc2a65c325dc2160ec2441e527f2e69a3e2b68abcf4be04c399e82f9bf6"} Jul 09 02:40:25.800980 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:25.800518 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lshzb/must-gather-xq6c8" event={"ID":"384a9156-93b8-433b-a6f1-c9a9899012d1","Type":"ContainerStarted","Data":"5bb5280572bb89be1e173809daf052a76dc556fe21215feca1e1dc3e576f38a3"} Jul 09 02:40:25.816986 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:25.816938 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lshzb/must-gather-xq6c8" podStartSLOduration=1.514078462 podStartE2EDuration="2.81692324s" totalImagesPullingTime="1.302844778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:40:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:40:25.815587139 +0000 UTC m=+2446.513640952" watchObservedRunningTime="2026-07-09 02:40:25.81692324 +0000 UTC m=+2446.514977074" Jul 09 02:40:26.590344 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:26.590313 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lg95f_7d15267e-8d0e-49e2-9a69-9c85f84b645f/global-pull-secret-syncer/0.log" Jul 09 02:40:26.650576 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:26.650540 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-k9d84_ae3b2d7f-4db7-486c-b226-b37e4cfab1f8/konnectivity-agent/0.log" Jul 09 02:40:26.727951 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:26.727911 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-184.ec2.internal_81d8756a43625938ca8b36e75b20e52a/haproxy/0.log" Jul 09 02:40:31.083438 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:31.083401 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l_34076dc0-01aa-4322-b5a4-ccf17bb46fad/extract/0.log" Jul 09 02:40:31.108204 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:31.108176 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l_34076dc0-01aa-4322-b5a4-ccf17bb46fad/util/0.log" Jul 09 02:40:31.127826 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:31.127789 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dnf9l_34076dc0-01aa-4322-b5a4-ccf17bb46fad/pull/0.log" Jul 09 02:40:31.174171 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:31.174140 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2_b19afe3c-311b-4c34-94ef-975696e39caf/extract/0.log" Jul 09 02:40:31.192427 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:31.192395 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2_b19afe3c-311b-4c34-94ef-975696e39caf/util/0.log" Jul 09 02:40:31.214263 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:31.213937 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ql6c2_b19afe3c-311b-4c34-94ef-975696e39caf/pull/0.log" Jul 09 02:40:31.240987 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:31.240948 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh_261f21fc-52be-4e8c-82ca-807f6ca6513d/extract/0.log" Jul 09 02:40:31.259772 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:31.259745 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh_261f21fc-52be-4e8c-82ca-807f6ca6513d/util/0.log" Jul 09 02:40:31.279335 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:31.279308 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mwtrh_261f21fc-52be-4e8c-82ca-807f6ca6513d/pull/0.log" Jul 09 02:40:31.304764 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:31.304733 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf_b859e0af-2e73-42e4-a3f4-36cc346aac92/extract/0.log" Jul 09 02:40:31.323203 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:31.323175 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf_b859e0af-2e73-42e4-a3f4-36cc346aac92/util/0.log" Jul 09 02:40:31.341679 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:31.341594 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gcdmf_b859e0af-2e73-42e4-a3f4-36cc346aac92/pull/0.log" Jul 09 02:40:31.645617 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:31.645590 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-x5tmh_eb5a1a9f-9cd5-4ebf-a070-ed56eec53f18/manager/0.log" Jul 09 02:40:31.703499 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:31.703468 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-lngv4_ca1cefe0-a5f8-4793-a7c6-ff21c7996095/manager/0.log" Jul 09 02:40:31.766780 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:31.766753 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-52mq2_9137fc0a-da4d-4f44-a462-32a6a81e8571/registry-server/0.log" Jul 09 02:40:31.932274 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:31.932190 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-rsbcx_c7f67406-2d81-48fa-8c0f-d3855f65eddf/manager/0.log" Jul 09 02:40:31.959513 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:31.959342 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-r7rtl_300bc103-5b60-4d66-9418-885fa3e554bb/limitador/0.log" Jul 09 02:40:32.063893 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:32.063831 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-q8zh5_8f5752c8-a347-415c-9cb4-5224c81bd875/manager/0.log" Jul 09 02:40:33.790014 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:33.789980 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-c4bd9748-29rrw_90b74197-f326-4f3f-97b0-f30b99287cdd/metrics-server/0.log" Jul 09 02:40:34.031850 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:34.031817 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ssczb_3640bda0-e436-4577-8c36-6661763d217a/node-exporter/0.log" Jul 09 02:40:34.055960 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:34.055865 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ssczb_3640bda0-e436-4577-8c36-6661763d217a/kube-rbac-proxy/0.log" Jul 09 02:40:34.078176 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:34.078142 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ssczb_3640bda0-e436-4577-8c36-6661763d217a/init-textfile/0.log" Jul 09 02:40:34.430233 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:34.430202 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-78cht_fa1c593a-8ea7-4290-8677-83e96fa7bd9a/prometheus-operator-admission-webhook/0.log" Jul 09 02:40:36.791065 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:36.791033 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl"] Jul 09 02:40:36.797313 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:36.797285 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:36.818061 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:36.818033 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl"] Jul 09 02:40:36.859553 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:36.859512 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7df91db-66fb-4b78-aae4-a641ad7ad70c-lib-modules\") pod \"perf-node-gather-daemonset-b58wl\" (UID: \"a7df91db-66fb-4b78-aae4-a641ad7ad70c\") " pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:36.859741 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:36.859633 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a7df91db-66fb-4b78-aae4-a641ad7ad70c-podres\") pod \"perf-node-gather-daemonset-b58wl\" (UID: \"a7df91db-66fb-4b78-aae4-a641ad7ad70c\") " pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:36.859807 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:36.859760 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7df91db-66fb-4b78-aae4-a641ad7ad70c-sys\") pod \"perf-node-gather-daemonset-b58wl\" (UID: \"a7df91db-66fb-4b78-aae4-a641ad7ad70c\") " pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:36.859807 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:36.859785 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a7df91db-66fb-4b78-aae4-a641ad7ad70c-proc\") pod \"perf-node-gather-daemonset-b58wl\" (UID: \"a7df91db-66fb-4b78-aae4-a641ad7ad70c\") " pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:36.859913 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:36.859807 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbtfz\" (UniqueName: \"kubernetes.io/projected/a7df91db-66fb-4b78-aae4-a641ad7ad70c-kube-api-access-wbtfz\") pod \"perf-node-gather-daemonset-b58wl\" (UID: \"a7df91db-66fb-4b78-aae4-a641ad7ad70c\") " pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:36.960681 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:36.960642 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a7df91db-66fb-4b78-aae4-a641ad7ad70c-podres\") pod \"perf-node-gather-daemonset-b58wl\" (UID: \"a7df91db-66fb-4b78-aae4-a641ad7ad70c\") " pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:36.960681 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:36.960694 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7df91db-66fb-4b78-aae4-a641ad7ad70c-sys\") pod \"perf-node-gather-daemonset-b58wl\" (UID: \"a7df91db-66fb-4b78-aae4-a641ad7ad70c\") " pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:36.960981 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:36.960719 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a7df91db-66fb-4b78-aae4-a641ad7ad70c-proc\") pod \"perf-node-gather-daemonset-b58wl\" (UID: \"a7df91db-66fb-4b78-aae4-a641ad7ad70c\") " pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:36.960981 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:36.960745 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbtfz\" (UniqueName: \"kubernetes.io/projected/a7df91db-66fb-4b78-aae4-a641ad7ad70c-kube-api-access-wbtfz\") pod \"perf-node-gather-daemonset-b58wl\" (UID: \"a7df91db-66fb-4b78-aae4-a641ad7ad70c\") " pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:36.960981 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:36.960827 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7df91db-66fb-4b78-aae4-a641ad7ad70c-lib-modules\") pod \"perf-node-gather-daemonset-b58wl\" (UID: \"a7df91db-66fb-4b78-aae4-a641ad7ad70c\") " pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:36.961249 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:36.961228 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a7df91db-66fb-4b78-aae4-a641ad7ad70c-proc\") pod \"perf-node-gather-daemonset-b58wl\" (UID: \"a7df91db-66fb-4b78-aae4-a641ad7ad70c\") " pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:36.961457 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:36.961441 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a7df91db-66fb-4b78-aae4-a641ad7ad70c-podres\") pod \"perf-node-gather-daemonset-b58wl\" (UID: \"a7df91db-66fb-4b78-aae4-a641ad7ad70c\") " pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:36.961598 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:36.961584 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7df91db-66fb-4b78-aae4-a641ad7ad70c-sys\") pod \"perf-node-gather-daemonset-b58wl\" (UID: \"a7df91db-66fb-4b78-aae4-a641ad7ad70c\") " pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:36.961788 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:36.961771 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7df91db-66fb-4b78-aae4-a641ad7ad70c-lib-modules\") pod \"perf-node-gather-daemonset-b58wl\" (UID: \"a7df91db-66fb-4b78-aae4-a641ad7ad70c\") " pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:36.970145 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:36.970122 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbtfz\" (UniqueName: \"kubernetes.io/projected/a7df91db-66fb-4b78-aae4-a641ad7ad70c-kube-api-access-wbtfz\") pod \"perf-node-gather-daemonset-b58wl\" (UID: \"a7df91db-66fb-4b78-aae4-a641ad7ad70c\") " pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:37.135585 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:37.135512 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:37.290112 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:37.290084 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl"] Jul 09 02:40:37.293309 ip-10-0-130-184 kubenswrapper[2582]: W0709 02:40:37.293280 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda7df91db_66fb_4b78_aae4_a641ad7ad70c.slice/crio-c8404af6f32b39b9feb95756ef5faabe3d04279680a27636d60d19efbfcc5c4b WatchSource:0}: Error finding container c8404af6f32b39b9feb95756ef5faabe3d04279680a27636d60d19efbfcc5c4b: Status 404 returned error can't find the container with id c8404af6f32b39b9feb95756ef5faabe3d04279680a27636d60d19efbfcc5c4b Jul 09 02:40:37.908742 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:37.908697 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" event={"ID":"a7df91db-66fb-4b78-aae4-a641ad7ad70c","Type":"ContainerStarted","Data":"4e15e0fbb986cb2aad3a9863a98ede25250a115571eca1b136756f023090bcdb"} Jul 09 02:40:37.909256 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:37.908771 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" event={"ID":"a7df91db-66fb-4b78-aae4-a641ad7ad70c","Type":"ContainerStarted","Data":"c8404af6f32b39b9feb95756ef5faabe3d04279680a27636d60d19efbfcc5c4b"} Jul 09 02:40:37.909683 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:37.909654 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:37.927403 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:37.927351 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" podStartSLOduration=1.927329491 podStartE2EDuration="1.927329491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 02:40:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:40:37.92467439 +0000 UTC m=+2458.622728203" watchObservedRunningTime="2026-07-09 02:40:37.927329491 +0000 UTC m=+2458.625383305" Jul 09 02:40:38.530392 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:38.530361 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2btw2_8a081582-6289-42d1-aa82-f7faea30f410/dns/0.log" Jul 09 02:40:38.548990 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:38.548957 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2btw2_8a081582-6289-42d1-aa82-f7faea30f410/kube-rbac-proxy/0.log" Jul 09 02:40:38.649833 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:38.649807 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bb6hq_2a403528-9b59-4685-974e-69e52cac4ec4/dns-node-resolver/0.log" Jul 09 02:40:39.205568 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:39.205535 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-57566d484-zxrgq_a1ddebc5-dddc-47b0-b213-c5f15172a8e2/registry/0.log" Jul 09 02:40:39.251107 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:39.251078 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qkgn2_83da97a5-3688-4c0d-97b6-cc3cc3ed3d25/node-ca/0.log" Jul 09 02:40:40.275705 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:40.275623 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-847fc5468-8lr4q_e3f31f39-5c9a-4fda-86fa-331f25d1f555/kube-auth-proxy/0.log" Jul 09 02:40:41.128853 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:41.128820 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ckv5l_91ecf239-3824-49c2-aaab-26dc1679fde1/serve-healthcheck-canary/0.log" Jul 09 02:40:41.704770 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:41.704733 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wrbh6_8495fb53-5899-4eb6-a4a1-9eca3914cdd1/kube-rbac-proxy/0.log" Jul 09 02:40:41.721765 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:41.721738 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wrbh6_8495fb53-5899-4eb6-a4a1-9eca3914cdd1/exporter/0.log" Jul 09 02:40:41.741322 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:41.741299 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wrbh6_8495fb53-5899-4eb6-a4a1-9eca3914cdd1/extractor/0.log" Jul 09 02:40:43.778289 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:43.778260 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-rkl5n_bf4f6c90-4abd-4907-b6cb-58b8f67af948/manager/0.log" Jul 09 02:40:44.042398 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:44.042304 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-dnrfl_433ba04e-75c0-4ce1-8793-aec0e8ff93d6/manager/0.log" Jul 09 02:40:44.221655 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:44.221621 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-669bbdcddd-9cp7q_27cf0c1a-9674-4132-b220-9bf4dc6834c1/manager/0.log" Jul 09 02:40:44.238181 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:44.238152 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-8cstw_fc976687-df22-4397-a0dd-797c604bc7a6/server/0.log" Jul 09 02:40:44.348865 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:44.348779 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2d6sg_06e1f2d4-cf8d-4d3b-8123-4741b828125f/manager/0.log" Jul 09 02:40:44.559377 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:44.559340 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8687897cbb-gbnt5_2c5cb7ea-589c-41ab-a5ca-54ada4284c7d/manager/0.log" Jul 09 02:40:44.574096 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:44.574064 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8687897cbb-gbnt5_2c5cb7ea-589c-41ab-a5ca-54ada4284c7d/copy-manifests/0.log" Jul 09 02:40:44.935404 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:44.935369 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lshzb/perf-node-gather-daemonset-b58wl" Jul 09 02:40:45.703691 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:45.703660 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-km4fn_8a6b58b4-f622-4762-b72b-21194f7cebc1/openshift-lws-operator/0.log" Jul 09 02:40:46.943149 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:46.943118 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-847fc5468-8lr4q_e3f31f39-5c9a-4fda-86fa-331f25d1f555/kube-auth-proxy/0.log" Jul 09 02:40:48.535289 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:48.535255 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-847fc5468-8lr4q_e3f31f39-5c9a-4fda-86fa-331f25d1f555/kube-auth-proxy/0.log" Jul 09 02:40:50.857954 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:50.857926 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-24dfr_3f541a57-ab39-4408-8578-3bf6d9d1f8f0/migrator/0.log" Jul 09 02:40:50.874845 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:50.874810 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-24dfr_3f541a57-ab39-4408-8578-3bf6d9d1f8f0/graceful-termination/0.log" Jul 09 02:40:52.196798 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:52.196773 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hktj_db97ff5f-15c4-4cef-b16c-fa03e39df587/kube-multus-additional-cni-plugins/0.log" Jul 09 02:40:52.213893 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:52.213848 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hktj_db97ff5f-15c4-4cef-b16c-fa03e39df587/egress-router-binary-copy/0.log" Jul 09 02:40:52.232202 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:52.232163 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hktj_db97ff5f-15c4-4cef-b16c-fa03e39df587/cni-plugins/0.log" Jul 09 02:40:52.250480 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:52.250460 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hktj_db97ff5f-15c4-4cef-b16c-fa03e39df587/bond-cni-plugin/0.log" Jul 09 02:40:52.266956 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:52.266935 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hktj_db97ff5f-15c4-4cef-b16c-fa03e39df587/routeoverride-cni/0.log" Jul 09 02:40:52.283429 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:52.283404 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hktj_db97ff5f-15c4-4cef-b16c-fa03e39df587/whereabouts-cni-bincopy/0.log" Jul 09 02:40:52.299647 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:52.299626 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hktj_db97ff5f-15c4-4cef-b16c-fa03e39df587/whereabouts-cni/0.log" Jul 09 02:40:52.660915 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:52.660884 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mmnsx_b3da9cb9-8c08-45b2-ad3e-5c2936c513a7/kube-multus/0.log" Jul 09 02:40:52.737836 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:52.737808 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-trtrf_c5e07105-f26f-4ea9-8cde-9db06c456edc/network-metrics-daemon/0.log" Jul 09 02:40:52.751021 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:52.751002 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-trtrf_c5e07105-f26f-4ea9-8cde-9db06c456edc/kube-rbac-proxy/0.log" Jul 09 02:40:53.703457 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:53.703427 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-controller/0.log" Jul 09 02:40:53.716520 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:53.716497 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/1.log" Jul 09 02:40:53.716704 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:53.716526 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovn-acl-logging/0.log" Jul 09 02:40:53.731776 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:53.731747 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/kube-rbac-proxy-node/0.log" Jul 09 02:40:53.749065 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:53.749042 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/kube-rbac-proxy-ovn-metrics/0.log" Jul 09 02:40:53.764980 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:53.764959 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/northd/0.log" Jul 09 02:40:53.783221 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:53.783196 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/nbdb/0.log" Jul 09 02:40:53.800225 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:53.800205 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/sbdb/0.log" Jul 09 02:40:53.877068 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:53.877039 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj28w_ec9d7c46-2a09-420b-90ae-5aad678928b0/ovnkube-controller/0.log" Jul 09 02:40:55.221346 ip-10-0-130-184 kubenswrapper[2582]: I0709 02:40:55.221317 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-989f8b8dc-l8gb2_0e2f0937-c209-4fe3-8715-6c36e4c6122e/check-endpoints/0.log"