Jul 13 17:05:22.196044 ip-10-0-137-245 systemd[1]: Starting Kubernetes Kubelet... Jul 13 17:05:22.707498 ip-10-0-137-245 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 17:05:22.707498 ip-10-0-137-245 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 17:05:22.707498 ip-10-0-137-245 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 17:05:22.707498 ip-10-0-137-245 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 17:05:22.707498 ip-10-0-137-245 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 17:05:22.710753 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.710647 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 17:05:22.716442 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716425 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 17:05:22.716442 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716442 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716446 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716449 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716452 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716455 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716458 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716461 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716463 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716466 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716469 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716471 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716474 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716478 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716487 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716490 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716493 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716496 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716499 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716501 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 17:05:22.716512 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716504 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716509 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716512 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716515 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716517 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716520 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716523 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716526 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716529 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716531 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716534 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716537 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716540 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716543 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716546 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716548 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716551 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716553 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716556 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716558 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 17:05:22.716992 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716561 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716563 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716567 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716569 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716574 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716577 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716580 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716582 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716585 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716588 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716591 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716593 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716596 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716598 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716601 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716604 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716608 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716612 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716615 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 17:05:22.717484 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716617 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716620 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716623 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716626 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716628 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716632 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716634 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716637 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716640 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716643 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716646 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716650 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716653 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716656 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716658 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716661 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716665 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716667 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716669 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716672 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 17:05:22.717968 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716674 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716677 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716679 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716682 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716684 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716687 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716689 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716694 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716697 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716699 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716702 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716705 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716707 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716710 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716713 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716715 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716718 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716720 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716722 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 17:05:22.718438 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.716736 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717461 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717469 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717472 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717475 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717478 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717480 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717483 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717486 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717488 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717492 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717495 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717498 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717501 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717503 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717506 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717509 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717511 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717514 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717517 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 17:05:22.718938 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717520 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717523 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717525 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717528 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717531 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717533 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717536 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717538 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717541 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717543 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717546 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717548 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717551 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717553 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717556 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717559 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717561 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717563 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717566 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717568 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717571 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 17:05:22.719464 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717573 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717576 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717579 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717581 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717584 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717586 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717590 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717594 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717597 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717600 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717603 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717606 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717608 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717611 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717614 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717616 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717618 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717621 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717624 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 17:05:22.719991 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717626 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717628 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717631 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717633 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717636 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717638 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717641 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717643 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717646 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717648 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717651 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717653 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717656 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717658 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717660 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717663 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717666 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717669 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717672 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 17:05:22.720462 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717674 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717677 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717679 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717685 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717688 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717691 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717694 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717697 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717700 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717702 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717705 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717708 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717710 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717714 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717718 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717721 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717723 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717744 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717747 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 17:05:22.720955 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717750 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.717752 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718167 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718172 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718175 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718178 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718181 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718183 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718185 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718188 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718192 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718195 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718198 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718200 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718203 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718206 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718208 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718211 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718214 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718217 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 17:05:22.721407 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718219 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718222 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718224 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718226 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718229 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718232 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718234 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718236 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718239 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718241 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718244 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718246 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718248 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718251 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718254 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718256 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718259 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718261 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718264 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718266 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 17:05:22.721900 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718269 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718271 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718274 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718277 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718279 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718281 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718284 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718286 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718288 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718291 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718294 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718297 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718299 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718302 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718304 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718307 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718309 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718312 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718314 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 17:05:22.722390 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718317 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718319 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718322 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718324 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718328 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718331 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718335 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718340 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718343 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718345 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718348 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718350 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718353 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718356 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718358 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718360 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718365 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718369 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718372 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 17:05:22.722876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718374 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718376 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718379 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718382 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718385 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718387 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718389 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718393 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718395 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718397 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718400 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718403 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718405 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718408 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718410 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718412 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718415 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718417 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718420 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718422 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 17:05:22.723418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718425 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718427 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.718429 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718505 2564 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718513 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718520 2564 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718524 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718531 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718539 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718546 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718553 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718558 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718561 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718565 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718568 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718571 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718574 2564 flags.go:64] FLAG: --cgroup-root="" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718577 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718580 2564 flags.go:64] FLAG: --client-ca-file="" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718582 2564 flags.go:64] FLAG: --cloud-provider="external" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718585 2564 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718589 2564 flags.go:64] FLAG: --cluster-domain="" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718592 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718595 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 17:05:22.723934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718598 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718601 2564 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718605 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718608 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718611 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718615 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718618 2564 flags.go:64] FLAG: --contention-profiling="false" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718621 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718624 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718626 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718629 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718633 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718636 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718639 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718642 2564 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718645 2564 flags.go:64] FLAG: --enable-server="true" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718649 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718654 2564 flags.go:64] FLAG: --event-burst="100" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718657 2564 flags.go:64] FLAG: --event-qps="50" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718660 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718663 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718666 2564 flags.go:64] FLAG: --eviction-hard="" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718670 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718673 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718676 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 17:05:22.724548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718678 2564 flags.go:64] FLAG: --eviction-soft="" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718681 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718684 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718687 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718689 2564 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718692 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718695 2564 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718702 2564 flags.go:64] FLAG: --feature-gates="" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718706 2564 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718709 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718713 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718716 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718719 2564 flags.go:64] FLAG: --healthz-port="10248" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718722 2564 flags.go:64] FLAG: --help="false" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718738 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-137-245.ec2.internal" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718742 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718745 2564 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718748 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718751 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718755 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718757 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718760 2564 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718763 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 17:05:22.725171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718767 2564 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718770 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718773 2564 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718776 2564 flags.go:64] FLAG: --kube-reserved="" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718778 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718781 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718785 2564 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718787 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718790 2564 flags.go:64] FLAG: --lock-file="" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718793 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718796 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718799 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718804 2564 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718807 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718810 2564 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718812 2564 flags.go:64] FLAG: --logging-format="text" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718817 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718820 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718823 2564 flags.go:64] FLAG: --manifest-url="" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718826 2564 flags.go:64] FLAG: --manifest-url-header="" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718831 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718834 2564 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718844 2564 flags.go:64] FLAG: --max-pods="110" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718847 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718850 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 17:05:22.725784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718853 2564 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718856 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718859 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718861 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718864 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718871 2564 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718874 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718879 2564 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718882 2564 flags.go:64] FLAG: --pod-cidr="" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718884 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718890 2564 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718893 2564 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718897 2564 flags.go:64] FLAG: --pods-per-core="0" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718900 2564 flags.go:64] FLAG: --port="10250" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718903 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718906 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02cf972a2e769b450" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718909 2564 flags.go:64] FLAG: --qos-reserved="" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718912 2564 flags.go:64] FLAG: --read-only-port="10255" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718914 2564 flags.go:64] FLAG: --register-node="true" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718918 2564 flags.go:64] FLAG: --register-with-taints="" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718921 2564 flags.go:64] FLAG: --registry-burst="10" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718924 2564 flags.go:64] FLAG: --registry-qps="5" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718927 2564 flags.go:64] FLAG: --reserved-cpus="" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718931 2564 flags.go:64] FLAG: --reserved-memory="" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718937 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 17:05:22.726404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718940 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718943 2564 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718946 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718949 2564 flags.go:64] FLAG: --runonce="false" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718952 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718955 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718958 2564 flags.go:64] FLAG: --seccomp-default="false" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718960 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718963 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718966 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718969 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718971 2564 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718974 2564 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718977 2564 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718979 2564 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718983 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718985 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718988 2564 flags.go:64] FLAG: --system-cgroups="" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718991 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718996 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.718998 2564 flags.go:64] FLAG: --tls-cert-file="" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.719001 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.719005 2564 flags.go:64] FLAG: --tls-min-version="" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.719008 2564 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.719011 2564 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 17:05:22.727276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.719014 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.719016 2564 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.719019 2564 flags.go:64] FLAG: --v="2" Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.719023 2564 flags.go:64] FLAG: --version="false" Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.719027 2564 flags.go:64] FLAG: --vmodule="" Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.719032 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.719035 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719153 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719156 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719159 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719163 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719165 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719168 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719170 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719173 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719175 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719178 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719180 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719183 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719185 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719188 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 17:05:22.728102 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719190 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719193 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719196 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719198 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719201 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719204 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719206 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719209 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719212 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719215 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719217 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719220 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719222 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719225 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719227 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719230 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719234 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719237 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719240 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 17:05:22.728614 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719244 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719247 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719250 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719252 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719255 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719257 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719260 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719262 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719264 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719267 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719270 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719273 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719275 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719278 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719281 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719283 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719287 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719290 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719293 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 17:05:22.729116 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719296 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719299 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719302 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719307 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719309 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719312 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719314 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719317 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719319 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719321 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719325 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719328 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719330 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719332 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719335 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719337 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719339 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719342 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719344 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719347 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 17:05:22.729576 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719349 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719351 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719354 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719356 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719359 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719361 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719364 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719366 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719369 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719371 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719374 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719376 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719379 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719381 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719383 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719386 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719389 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719391 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719393 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719396 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 17:05:22.730093 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719398 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 17:05:22.730566 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719400 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 17:05:22.730566 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719404 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 17:05:22.730566 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719406 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 17:05:22.730566 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719409 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 17:05:22.730566 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719411 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 17:05:22.730566 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.719413 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 17:05:22.730566 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.720195 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 17:05:22.730566 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.729600 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 17:05:22.730566 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.729618 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 17:05:22.730566 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.729639 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 17:05:22.730566 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.729646 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 17:05:22.730566 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729704 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 17:05:22.730566 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729709 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 13 17:05:22.730566 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729713 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 17:05:22.730566 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729716 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729719 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729721 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729742 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729746 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729750 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729753 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729755 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729758 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729761 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729764 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729767 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729770 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729773 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729775 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729778 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729781 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729784 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729786 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729789 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 17:05:22.730986 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729792 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729794 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729798 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729801 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729804 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729806 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729809 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729812 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729814 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729817 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729819 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729822 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729825 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729827 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729829 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729832 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729835 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729837 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729840 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729842 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 17:05:22.731467 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729845 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729847 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729850 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729852 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729856 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729860 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729863 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729866 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729869 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729871 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729874 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729877 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729880 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729882 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729884 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729887 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729889 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729892 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729895 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 17:05:22.732001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729898 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729900 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729903 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729906 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729910 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729913 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729916 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729918 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729921 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729924 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729926 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729929 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729931 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729935 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729939 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729942 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729944 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729947 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729949 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 17:05:22.732452 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729952 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 17:05:22.732941 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729954 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 17:05:22.732941 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729957 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 17:05:22.732941 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729959 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 17:05:22.732941 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729962 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 17:05:22.732941 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729965 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 17:05:22.732941 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729967 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 17:05:22.732941 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729970 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 17:05:22.732941 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729973 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 17:05:22.732941 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729975 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 17:05:22.732941 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729977 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 17:05:22.732941 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729980 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 17:05:22.732941 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729983 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 17:05:22.732941 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729985 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 17:05:22.732941 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729988 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 17:05:22.732941 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729991 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 17:05:22.732941 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729993 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 17:05:22.732941 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.729995 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 17:05:22.733349 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.730001 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 17:05:22.733349 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730115 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 17:05:22.733349 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730120 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 17:05:22.733349 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730123 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 17:05:22.733349 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730126 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 17:05:22.733349 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730129 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 17:05:22.733349 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730132 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 17:05:22.733349 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730135 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 17:05:22.733349 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730138 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 17:05:22.733349 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730140 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 17:05:22.733349 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730143 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 17:05:22.733349 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730146 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 17:05:22.733349 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730148 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 17:05:22.733349 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730151 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 17:05:22.733349 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730154 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730157 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730160 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730162 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730166 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730169 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730171 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730174 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730178 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730182 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730185 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730188 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730191 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730193 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730196 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730199 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730201 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730204 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730207 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730209 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 17:05:22.733701 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730212 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730214 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730217 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730219 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730221 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730224 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730226 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730228 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730231 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730233 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730236 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730239 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730242 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730244 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730247 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730249 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730253 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730256 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730259 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 17:05:22.734209 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730261 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730264 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730267 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730270 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730273 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730276 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730279 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730281 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730284 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730286 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730288 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730291 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730293 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730296 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730298 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730300 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730302 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730305 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730307 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730310 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 17:05:22.734670 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730312 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730315 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730317 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730319 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730322 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730324 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730328 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730331 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730333 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730335 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730338 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730340 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730342 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730345 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730347 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730349 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730352 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730355 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730357 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730360 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 17:05:22.735192 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730362 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 17:05:22.735662 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730365 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 17:05:22.735662 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730367 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 17:05:22.735662 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730369 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 17:05:22.735662 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730372 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 17:05:22.735662 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730374 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 17:05:22.735662 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:22.730377 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 17:05:22.735662 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.730382 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 17:05:22.735662 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.730503 2564 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 17:05:22.735662 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.735045 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 17:05:22.736857 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.736844 2564 server.go:1019] "Starting client certificate rotation" Jul 13 17:05:22.736969 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.736951 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 17:05:22.737004 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.736995 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 17:05:22.741217 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.741203 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 17:05:22.744867 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.744844 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 17:05:22.763106 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.763083 2564 log.go:25] "Validated CRI v1 runtime API" Jul 13 17:05:22.765400 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.765383 2564 log.go:25] "Validated CRI v1 image API" Jul 13 17:05:22.766208 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.766184 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 17:05:22.766934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.766910 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 17:05:22.769287 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.769268 2564 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c8851a99-9689-4b6a-bdd9-4a34dd2bded7:/dev/nvme0n1p3 ccb089db-c034-41d1-b784-0df403543d74:/dev/nvme0n1p4] Jul 13 17:05:22.769350 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.769287 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 17:05:22.783148 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.783010 2564 manager.go:216] Machine: {Timestamp:2026-07-13 17:05:22.780564686 +0000 UTC m=+0.443339867 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3134986 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2eb9e96b0e317bba21ee164c54c45c SystemUUID:ec2eb9e9-6b0e-317b-ba21-ee164c54c45c BootID:cdf3e128-4937-431f-9e36-20dbccb512b7 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9c:e9:02:4f:0f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9c:e9:02:4f:0f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:9b:93:dd:f9:ce Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 17:05:22.783148 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.783139 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 17:05:22.783294 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.783230 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 17:05:22.787486 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.787459 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 17:05:22.787626 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.787490 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-245.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 17:05:22.788385 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.788374 2564 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 17:05:22.788420 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.788388 2564 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 17:05:22.788420 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.788396 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 17:05:22.788420 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.788411 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 17:05:22.789296 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.789285 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 17:05:22.790371 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.790354 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xq4kl" Jul 13 17:05:22.793164 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.793152 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 13 17:05:22.793293 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.793283 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 17:05:22.793329 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.793318 2564 kubelet.go:486] "Attempting to sync node with API server" Jul 13 17:05:22.793356 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.793331 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 17:05:22.793356 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.793351 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 17:05:22.793406 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.793361 2564 kubelet.go:398] "Adding apiserver pod source" Jul 13 17:05:22.793406 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.793378 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 17:05:22.793764 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.793752 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 17:05:22.793801 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.793772 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 17:05:22.796360 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.796344 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 17:05:22.797629 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.797608 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xq4kl" Jul 13 17:05:22.797818 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.797806 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 17:05:22.797854 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.797835 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 17:05:22.798498 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.798483 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 17:05:22.798532 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.798508 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 17:05:22.798532 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.798517 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 17:05:22.798532 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.798523 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 17:05:22.798532 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.798531 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 17:05:22.798689 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.798537 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 17:05:22.799193 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.799183 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 17:05:22.799224 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.799195 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 17:05:22.799224 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.799201 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 17:05:22.799224 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.799207 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 17:05:22.799224 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.799215 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 17:05:22.799224 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.799223 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 17:05:22.801399 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.801385 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 17:05:22.801399 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.801400 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 17:05:22.805024 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.805009 2564 server.go:1268] "Started kubelet" Jul 13 17:05:22.805136 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.805088 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 17:05:22.805248 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.805201 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 17:05:22.805311 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.805269 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 17:05:22.806593 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.806572 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 17:05:22.806780 ip-10-0-137-245 systemd[1]: Started Kubernetes Kubelet. Jul 13 17:05:22.807699 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.807639 2564 server.go:310] "Adding debug handlers to kubelet server" Jul 13 17:05:22.808214 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.808194 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 17:05:22.808999 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.808973 2564 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-137-245.ec2.internal" not found Jul 13 17:05:22.810628 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.810609 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 17:05:22.811567 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:22.811548 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 17:05:22.814815 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.814795 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 17:05:22.814897 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.814812 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 17:05:22.814897 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:22.814879 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-245.ec2.internal\" not found" Jul 13 17:05:22.815499 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.815481 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 17:05:22.815610 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.815501 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 17:05:22.815610 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.815520 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 17:05:22.815718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.815642 2564 reconstruct.go:97] "Volume reconstruction finished" Jul 13 17:05:22.815718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.815655 2564 reconciler.go:29] "Reconciler: start to sync state" Jul 13 17:05:22.815960 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.815944 2564 factory.go:55] Registering systemd factory Jul 13 17:05:22.816006 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.815963 2564 factory.go:223] Registration of the systemd container factory successfully Jul 13 17:05:22.816163 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.816152 2564 factory.go:153] Registering CRI-O factory Jul 13 17:05:22.816216 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.816169 2564 factory.go:223] Registration of the crio container factory successfully Jul 13 17:05:22.816259 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.816226 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 17:05:22.816259 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.816253 2564 factory.go:103] Registering Raw factory Jul 13 17:05:22.816312 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.816268 2564 manager.go:1196] Started watching for new ooms in manager Jul 13 17:05:22.816673 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.816661 2564 manager.go:349] Starting recovery of all containers Jul 13 17:05:22.819092 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.816965 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 17:05:22.819092 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:22.818921 2564 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-245.ec2.internal\" not found" node="ip-10-0-137-245.ec2.internal" Jul 13 17:05:22.823606 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.823452 2564 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-137-245.ec2.internal" not found Jul 13 17:05:22.825135 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.825119 2564 manager.go:354] Recovery completed Jul 13 17:05:22.829424 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.829407 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 17:05:22.831428 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.831410 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-245.ec2.internal" event="NodeHasSufficientMemory" Jul 13 17:05:22.831498 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.831446 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-245.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 17:05:22.831498 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.831458 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-245.ec2.internal" event="NodeHasSufficientPID" Jul 13 17:05:22.832090 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.832076 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 17:05:22.832172 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.832093 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 17:05:22.832172 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.832113 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 13 17:05:22.832172 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.832084 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 17:05:22.832882 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.832867 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-245.ec2.internal" event="NodeHasSufficientMemory" Jul 13 17:05:22.832955 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.832899 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-245.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 17:05:22.832955 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.832913 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-245.ec2.internal" event="NodeHasSufficientPID" Jul 13 17:05:22.833575 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.833562 2564 policy_none.go:49] "None policy: Start" Jul 13 17:05:22.833620 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.833580 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 17:05:22.833620 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.833590 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 17:05:22.834671 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.834661 2564 policy_none.go:47] "Start" Jul 13 17:05:22.873368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.873355 2564 manager.go:337] "Starting Device Plugin manager" Jul 13 17:05:22.892953 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:22.873393 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 17:05:22.892953 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.873403 2564 server.go:87] "Starting device plugin registration server" Jul 13 17:05:22.892953 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.873627 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 17:05:22.892953 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.873640 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 17:05:22.892953 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.873743 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 17:05:22.892953 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.873855 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 17:05:22.892953 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.873865 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 17:05:22.892953 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:22.876645 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 17:05:22.892953 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:22.876701 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-245.ec2.internal\" not found" Jul 13 17:05:22.892953 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.878531 2564 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-137-245.ec2.internal" not found Jul 13 17:05:22.952888 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.952855 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 17:05:22.954290 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.954270 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 17:05:22.954381 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.954329 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 17:05:22.954381 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.954354 2564 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 17:05:22.954458 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:22.954396 2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 13 17:05:22.957021 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.956982 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 17:05:22.974550 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.974485 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 17:05:22.975531 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.975512 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-245.ec2.internal" event="NodeHasSufficientMemory" Jul 13 17:05:22.975614 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.975546 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-245.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 17:05:22.975614 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.975556 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-245.ec2.internal" event="NodeHasSufficientPID" Jul 13 17:05:22.975614 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.975575 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-245.ec2.internal" Jul 13 17:05:22.984795 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:22.984767 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-245.ec2.internal" Jul 13 17:05:23.056290 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.055428 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-245.ec2.internal"] Jul 13 17:05:23.057951 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.057932 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal" Jul 13 17:05:23.059097 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.059073 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-245.ec2.internal" Jul 13 17:05:23.092368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.092347 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal" Jul 13 17:05:23.096847 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.096828 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-245.ec2.internal" Jul 13 17:05:23.113087 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.113064 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 17:05:23.113179 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.113164 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 17:05:23.217022 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.216992 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/abc7e90015ad279ad8efd2a082fbdc6c-config\") pod \"kube-apiserver-proxy-ip-10-0-137-245.ec2.internal\" (UID: \"abc7e90015ad279ad8efd2a082fbdc6c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-245.ec2.internal" Jul 13 17:05:23.217022 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.217024 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0cdaf1109a9a8f4611e430e48feed9fc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal\" (UID: \"0cdaf1109a9a8f4611e430e48feed9fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal" Jul 13 17:05:23.217194 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.217045 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0cdaf1109a9a8f4611e430e48feed9fc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal\" (UID: \"0cdaf1109a9a8f4611e430e48feed9fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal" Jul 13 17:05:23.318190 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.318099 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0cdaf1109a9a8f4611e430e48feed9fc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal\" (UID: \"0cdaf1109a9a8f4611e430e48feed9fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal" Jul 13 17:05:23.318190 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.318133 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0cdaf1109a9a8f4611e430e48feed9fc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal\" (UID: \"0cdaf1109a9a8f4611e430e48feed9fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal" Jul 13 17:05:23.318190 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.318155 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/abc7e90015ad279ad8efd2a082fbdc6c-config\") pod \"kube-apiserver-proxy-ip-10-0-137-245.ec2.internal\" (UID: \"abc7e90015ad279ad8efd2a082fbdc6c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-245.ec2.internal" Jul 13 17:05:23.318396 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.318204 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/abc7e90015ad279ad8efd2a082fbdc6c-config\") pod \"kube-apiserver-proxy-ip-10-0-137-245.ec2.internal\" (UID: \"abc7e90015ad279ad8efd2a082fbdc6c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-245.ec2.internal" Jul 13 17:05:23.318396 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.318210 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0cdaf1109a9a8f4611e430e48feed9fc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal\" (UID: \"0cdaf1109a9a8f4611e430e48feed9fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal" Jul 13 17:05:23.318396 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.318207 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0cdaf1109a9a8f4611e430e48feed9fc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal\" (UID: \"0cdaf1109a9a8f4611e430e48feed9fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal" Jul 13 17:05:23.416554 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.416523 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal" Jul 13 17:05:23.417592 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.417566 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-245.ec2.internal" Jul 13 17:05:23.736060 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.735975 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 17:05:23.736822 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.736124 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 17:05:23.736822 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.736148 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 17:05:23.736822 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.736174 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 17:05:23.793885 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.793859 2564 apiserver.go:52] "Watching apiserver" Jul 13 17:05:23.801800 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.801752 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 17:00:22 +0000 UTC" deadline="2028-04-01 16:07:53.886113808 +0000 UTC" Jul 13 17:05:23.801800 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.801796 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15071h2m30.084320494s" Jul 13 17:05:23.803158 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.803142 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 17:05:23.806493 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.806459 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-bm8v7","openshift-image-registry/node-ca-vq8xq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal","openshift-network-operator/iptables-alerter-f9mq2","openshift-ovn-kubernetes/ovnkube-node-wmxtg","kube-system/global-pull-secret-syncer-kj2dr","kube-system/konnectivity-agent-c8xq8","kube-system/kube-apiserver-proxy-ip-10-0-137-245.ec2.internal","openshift-cluster-node-tuning-operator/tuned-frz8b","openshift-multus/multus-4fwz6","openshift-multus/multus-additional-cni-plugins-wc8ng","openshift-multus/network-metrics-daemon-vpn5b","openshift-network-diagnostics/network-check-target-jz68h","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q"] Jul 13 17:05:23.808208 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.808192 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c8xq8" Jul 13 17:05:23.809436 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.809413 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vq8xq" Jul 13 17:05:23.810716 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.810699 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.811176 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.811161 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 17:05:23.811484 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.811465 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cgb22\"" Jul 13 17:05:23.811562 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.811467 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 17:05:23.812024 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.812009 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f9mq2" Jul 13 17:05:23.812451 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.812435 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 17:05:23.812548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.812534 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 17:05:23.812642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.812614 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-72vs6\"" Jul 13 17:05:23.812813 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.812799 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 17:05:23.813344 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.813328 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.813661 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.813630 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-blvts\"" Jul 13 17:05:23.813775 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.813760 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 17:05:23.813850 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.813835 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 17:05:23.813892 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.813878 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 17:05:23.814542 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.814526 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 17:05:23.814542 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.814538 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 17:05:23.814756 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.814744 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:23.814815 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:23.814798 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpn5b" podUID="738b8868-dacb-4206-bb1f-cbc6fdb57ecd" Jul 13 17:05:23.815055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.815041 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cjs7z\"" Jul 13 17:05:23.815128 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.815031 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 17:05:23.815238 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.815224 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 17:05:23.817721 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.817635 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 17:05:23.817721 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.817641 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 17:05:23.817901 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.817763 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 17:05:23.817901 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.817788 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 17:05:23.817901 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.817803 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 17:05:23.818016 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.817983 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 17:05:23.818084 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.818065 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.818791 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.818755 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fnbwf\"" Jul 13 17:05:23.820193 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820173 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-multus-conf-dir\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.820306 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820220 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlpwf\" (UniqueName: \"kubernetes.io/projected/c891937a-13fe-41d6-b824-2d569278dedb-kube-api-access-tlpwf\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.820306 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820246 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c891937a-13fe-41d6-b824-2d569278dedb-multus-daemon-config\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.820306 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820282 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-systemd-units\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.820449 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820323 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-registration-dir\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.820449 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820392 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slj4v\" (UniqueName: \"kubernetes.io/projected/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-kube-api-access-slj4v\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.820449 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820428 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/55262c75-f4d2-4f27-a369-e599c35d58a5-iptables-alerter-script\") pod \"iptables-alerter-f9mq2\" (UID: \"55262c75-f4d2-4f27-a369-e599c35d58a5\") " pod="openshift-network-operator/iptables-alerter-f9mq2" Jul 13 17:05:23.820560 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820459 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-multus-socket-dir-parent\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.820560 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820493 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-hostroot\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.820560 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820534 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-node-log\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.820661 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820560 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9nfcs\"" Jul 13 17:05:23.820661 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820570 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-os-release\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.820661 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820596 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-sys-fs\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.820661 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820618 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-kubelet\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.820661 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820643 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/376f8f23-3eb3-46bc-aaec-7bd8007fe531-ovnkube-script-lib\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.820923 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820665 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-etc-kubernetes\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.820923 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820703 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-host-run-netns\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.820923 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820719 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/376f8f23-3eb3-46bc-aaec-7bd8007fe531-env-overrides\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.820923 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820755 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/71c06287-e098-4864-9a00-fc294a063aa0-agent-certs\") pod \"konnectivity-agent-c8xq8\" (UID: \"71c06287-e098-4864-9a00-fc294a063aa0\") " pod="kube-system/konnectivity-agent-c8xq8" Jul 13 17:05:23.820923 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820777 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:23.820923 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820776 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ab1216cc-4528-474c-a329-daefce87939c-serviceca\") pod \"node-ca-vq8xq\" (UID: \"ab1216cc-4528-474c-a329-daefce87939c\") " pod="openshift-image-registry/node-ca-vq8xq" Jul 13 17:05:23.820923 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:23.820837 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz68h" podUID="cbe6b915-efe6-4616-b39e-cc1066ebd8e4" Jul 13 17:05:23.820923 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820871 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-host-run-k8s-cni-cncf-io\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.820923 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820900 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29b62\" (UniqueName: \"kubernetes.io/projected/ab1216cc-4528-474c-a329-daefce87939c-kube-api-access-29b62\") pod \"node-ca-vq8xq\" (UID: \"ab1216cc-4528-474c-a329-daefce87939c\") " pod="openshift-image-registry/node-ca-vq8xq" Jul 13 17:05:23.820923 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820924 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sf7s\" (UniqueName: \"kubernetes.io/projected/55262c75-f4d2-4f27-a369-e599c35d58a5-kube-api-access-6sf7s\") pod \"iptables-alerter-f9mq2\" (UID: \"55262c75-f4d2-4f27-a369-e599c35d58a5\") " pod="openshift-network-operator/iptables-alerter-f9mq2" Jul 13 17:05:23.821368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820940 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-slash\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.821368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820955 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/376f8f23-3eb3-46bc-aaec-7bd8007fe531-ovnkube-config\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.821368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820969 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzdkb\" (UniqueName: \"kubernetes.io/projected/376f8f23-3eb3-46bc-aaec-7bd8007fe531-kube-api-access-fzdkb\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.821368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.820986 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs\") pod \"network-metrics-daemon-vpn5b\" (UID: \"738b8868-dacb-4206-bb1f-cbc6fdb57ecd\") " pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:23.821368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821005 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c891937a-13fe-41d6-b824-2d569278dedb-cni-binary-copy\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.821368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821019 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-cni-netd\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.821368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821033 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ab1216cc-4528-474c-a329-daefce87939c-host\") pod \"node-ca-vq8xq\" (UID: \"ab1216cc-4528-474c-a329-daefce87939c\") " pod="openshift-image-registry/node-ca-vq8xq" Jul 13 17:05:23.821368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821046 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-host-var-lib-kubelet\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.821368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821068 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-run-netns\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.821368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821084 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-run-ovn-kubernetes\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.821368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821097 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt9dj\" (UniqueName: \"kubernetes.io/projected/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-kube-api-access-rt9dj\") pod \"network-metrics-daemon-vpn5b\" (UID: \"738b8868-dacb-4206-bb1f-cbc6fdb57ecd\") " pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:23.821368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821112 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-host-var-lib-cni-bin\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.821368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821127 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.821368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821149 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.821368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821150 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 17:05:23.821368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821164 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-device-dir\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.821368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821186 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-etc-selinux\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.822127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821199 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-run-systemd\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.822127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821214 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-etc-openvswitch\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.822127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821232 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-run-openvswitch\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.822127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821246 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/376f8f23-3eb3-46bc-aaec-7bd8007fe531-ovn-node-metrics-cert\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.822127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821260 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-socket-dir\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.822127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821274 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55262c75-f4d2-4f27-a369-e599c35d58a5-host-slash\") pod \"iptables-alerter-f9mq2\" (UID: \"55262c75-f4d2-4f27-a369-e599c35d58a5\") " pod="openshift-network-operator/iptables-alerter-f9mq2" Jul 13 17:05:23.822127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821289 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-system-cni-dir\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.822127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821308 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-host-run-multus-certs\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.822127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821340 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-run-ovn\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.822127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821368 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-cnibin\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.822127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821097 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 17:05:23.822127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821429 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-var-lib-openvswitch\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.822127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821464 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-cni-bin\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.822127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821493 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/71c06287-e098-4864-9a00-fc294a063aa0-konnectivity-ca\") pod \"konnectivity-agent-c8xq8\" (UID: \"71c06287-e098-4864-9a00-fc294a063aa0\") " pod="kube-system/konnectivity-agent-c8xq8" Jul 13 17:05:23.822127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821509 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-multus-cni-dir\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.822127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821527 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-log-socket\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.822127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.821548 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-host-var-lib-cni-multus\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.822127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.822074 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 17:05:23.822759 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.822180 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bm8v7" Jul 13 17:05:23.822824 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.822809 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 17:05:23.823824 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.823805 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:23.824589 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.824571 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 17:05:23.824955 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.824936 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 17:05:23.825048 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.824988 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7nfkq\"" Jul 13 17:05:23.825306 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.825291 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:23.826686 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.826670 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 17:05:23.826791 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.826674 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:23.826791 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.826785 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 17:05:23.826905 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.826751 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gcb9k\"" Jul 13 17:05:23.826905 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:23.826812 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj2dr" podUID="8dcab0f9-5abb-4fa7-af28-54be545eb542" Jul 13 17:05:23.828367 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.828347 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 17:05:23.828451 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.828382 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kqcv6\"" Jul 13 17:05:23.828451 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.828363 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 17:05:23.833865 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.833841 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 17:05:23.857188 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.857170 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zq5nx" Jul 13 17:05:23.865250 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.865230 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zq5nx" Jul 13 17:05:23.916834 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.916790 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 17:05:23.921813 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.921786 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/71c06287-e098-4864-9a00-fc294a063aa0-konnectivity-ca\") pod \"konnectivity-agent-c8xq8\" (UID: \"71c06287-e098-4864-9a00-fc294a063aa0\") " pod="kube-system/konnectivity-agent-c8xq8" Jul 13 17:05:23.921948 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.921831 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-multus-cni-dir\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.921948 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.921865 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-modprobe-d\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:23.921948 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.921894 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-log-socket\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.921948 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.921919 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-host-var-lib-cni-multus\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.921948 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.921941 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-multus-conf-dir\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.922195 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.921963 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlpwf\" (UniqueName: \"kubernetes.io/projected/c891937a-13fe-41d6-b824-2d569278dedb-kube-api-access-tlpwf\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.922195 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.921992 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-multus-cni-dir\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.922195 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922000 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-log-socket\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.922195 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922021 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-host-var-lib-cni-multus\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.922195 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922050 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d-hosts-file\") pod \"node-resolver-bm8v7\" (UID: \"33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d\") " pod="openshift-dns/node-resolver-bm8v7" Jul 13 17:05:23.922195 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922069 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-multus-conf-dir\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.922195 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922082 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxllm\" (UniqueName: \"kubernetes.io/projected/e6411f69-741b-44f2-ae0f-7babdd39a621-kube-api-access-vxllm\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:23.922195 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922112 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c891937a-13fe-41d6-b824-2d569278dedb-multus-daemon-config\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.922195 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922178 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-systemd-units\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.922619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922211 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-registration-dir\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.922619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922238 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slj4v\" (UniqueName: \"kubernetes.io/projected/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-kube-api-access-slj4v\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.922619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922262 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/55262c75-f4d2-4f27-a369-e599c35d58a5-iptables-alerter-script\") pod \"iptables-alerter-f9mq2\" (UID: \"55262c75-f4d2-4f27-a369-e599c35d58a5\") " pod="openshift-network-operator/iptables-alerter-f9mq2" Jul 13 17:05:23.922619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922267 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-systemd-units\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.922619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922287 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-multus-socket-dir-parent\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.922619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922311 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-hostroot\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.922619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922341 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e6411f69-741b-44f2-ae0f-7babdd39a621-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:23.922619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922360 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/71c06287-e098-4864-9a00-fc294a063aa0-konnectivity-ca\") pod \"konnectivity-agent-c8xq8\" (UID: \"71c06287-e098-4864-9a00-fc294a063aa0\") " pod="kube-system/konnectivity-agent-c8xq8" Jul 13 17:05:23.922619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922427 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-node-log\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.922619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922368 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-node-log\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.922619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922457 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-os-release\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.922619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922478 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-tuned\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:23.922619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922495 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8dcab0f9-5abb-4fa7-af28-54be545eb542-dbus\") pod \"global-pull-secret-syncer-kj2dr\" (UID: \"8dcab0f9-5abb-4fa7-af28-54be545eb542\") " pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:23.922619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922518 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-registration-dir\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.922619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922537 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-hostroot\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.922619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922554 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-os-release\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.922619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922582 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-multus-socket-dir-parent\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.923280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922585 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-sys-fs\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.923280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922617 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-sys-fs\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.923280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922624 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-kubelet\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.923280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922651 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/376f8f23-3eb3-46bc-aaec-7bd8007fe531-ovnkube-script-lib\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.923280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922675 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-etc-kubernetes\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.923280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922699 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-kubelet\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.923280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922704 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c891937a-13fe-41d6-b824-2d569278dedb-multus-daemon-config\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.923280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922759 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-host-run-netns\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.923280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922783 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-systemd\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:23.923280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922791 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-host-run-netns\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.923280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922808 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/376f8f23-3eb3-46bc-aaec-7bd8007fe531-env-overrides\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.923280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922824 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/71c06287-e098-4864-9a00-fc294a063aa0-agent-certs\") pod \"konnectivity-agent-c8xq8\" (UID: \"71c06287-e098-4864-9a00-fc294a063aa0\") " pod="kube-system/konnectivity-agent-c8xq8" Jul 13 17:05:23.923280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922759 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-etc-kubernetes\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.923280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922839 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ab1216cc-4528-474c-a329-daefce87939c-serviceca\") pod \"node-ca-vq8xq\" (UID: \"ab1216cc-4528-474c-a329-daefce87939c\") " pod="openshift-image-registry/node-ca-vq8xq" Jul 13 17:05:23.923280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922878 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-host-run-k8s-cni-cncf-io\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.923280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922902 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-sysctl-conf\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:23.923280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922914 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/55262c75-f4d2-4f27-a369-e599c35d58a5-iptables-alerter-script\") pod \"iptables-alerter-f9mq2\" (UID: \"55262c75-f4d2-4f27-a369-e599c35d58a5\") " pod="openshift-network-operator/iptables-alerter-f9mq2" Jul 13 17:05:23.923280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922926 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29b62\" (UniqueName: \"kubernetes.io/projected/ab1216cc-4528-474c-a329-daefce87939c-kube-api-access-29b62\") pod \"node-ca-vq8xq\" (UID: \"ab1216cc-4528-474c-a329-daefce87939c\") " pod="openshift-image-registry/node-ca-vq8xq" Jul 13 17:05:23.924144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922936 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-host-run-k8s-cni-cncf-io\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.924144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922950 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sf7s\" (UniqueName: \"kubernetes.io/projected/55262c75-f4d2-4f27-a369-e599c35d58a5-kube-api-access-6sf7s\") pod \"iptables-alerter-f9mq2\" (UID: \"55262c75-f4d2-4f27-a369-e599c35d58a5\") " pod="openshift-network-operator/iptables-alerter-f9mq2" Jul 13 17:05:23.924144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.922973 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvjs5\" (UniqueName: \"kubernetes.io/projected/aab50e53-03f9-49ee-8382-c16c7ee45cf2-kube-api-access-vvjs5\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:23.924144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923192 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 17:05:23.924144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923247 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ab1216cc-4528-474c-a329-daefce87939c-serviceca\") pod \"node-ca-vq8xq\" (UID: \"ab1216cc-4528-474c-a329-daefce87939c\") " pod="openshift-image-registry/node-ca-vq8xq" Jul 13 17:05:23.924144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923209 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/376f8f23-3eb3-46bc-aaec-7bd8007fe531-env-overrides\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.924144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923270 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/376f8f23-3eb3-46bc-aaec-7bd8007fe531-ovnkube-script-lib\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.924144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923468 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-slash\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.924144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923493 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/376f8f23-3eb3-46bc-aaec-7bd8007fe531-ovnkube-config\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.924144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923536 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzdkb\" (UniqueName: \"kubernetes.io/projected/376f8f23-3eb3-46bc-aaec-7bd8007fe531-kube-api-access-fzdkb\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.924144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923568 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-slash\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.924144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923570 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs\") pod \"network-metrics-daemon-vpn5b\" (UID: \"738b8868-dacb-4206-bb1f-cbc6fdb57ecd\") " pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:23.924144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923613 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c891937a-13fe-41d6-b824-2d569278dedb-cni-binary-copy\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.924144 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:23.923643 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:23.924144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923653 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d-tmp-dir\") pod \"node-resolver-bm8v7\" (UID: \"33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d\") " pod="openshift-dns/node-resolver-bm8v7" Jul 13 17:05:23.924144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923682 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-sysconfig\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:23.924144 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:23.923707 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs podName:738b8868-dacb-4206-bb1f-cbc6fdb57ecd nodeName:}" failed. No retries permitted until 2026-07-13 17:05:24.423688351 +0000 UTC m=+2.086463556 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs") pod "network-metrics-daemon-vpn5b" (UID: "738b8868-dacb-4206-bb1f-cbc6fdb57ecd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:23.924718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923751 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-run\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:23.924718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923775 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-cni-netd\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.924718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923793 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ab1216cc-4528-474c-a329-daefce87939c-host\") pod \"node-ca-vq8xq\" (UID: \"ab1216cc-4528-474c-a329-daefce87939c\") " pod="openshift-image-registry/node-ca-vq8xq" Jul 13 17:05:23.924718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923813 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-host-var-lib-kubelet\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.924718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923847 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ab1216cc-4528-474c-a329-daefce87939c-host\") pod \"node-ca-vq8xq\" (UID: \"ab1216cc-4528-474c-a329-daefce87939c\") " pod="openshift-image-registry/node-ca-vq8xq" Jul 13 17:05:23.924718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923881 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-cni-netd\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.924718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923924 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-host-var-lib-kubelet\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.924718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923930 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qz9w\" (UniqueName: \"kubernetes.io/projected/33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d-kube-api-access-2qz9w\") pod \"node-resolver-bm8v7\" (UID: \"33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d\") " pod="openshift-dns/node-resolver-bm8v7" Jul 13 17:05:23.924718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923937 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/376f8f23-3eb3-46bc-aaec-7bd8007fe531-ovnkube-config\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.924718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.923962 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-sysctl-d\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:23.924718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924001 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-lib-modules\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:23.924718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924037 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6411f69-741b-44f2-ae0f-7babdd39a621-system-cni-dir\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:23.924718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924067 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e6411f69-741b-44f2-ae0f-7babdd39a621-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:23.924718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924090 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c891937a-13fe-41d6-b824-2d569278dedb-cni-binary-copy\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.924718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924099 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-run-netns\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.924718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924131 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-run-ovn-kubernetes\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.924718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924142 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-run-netns\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.925642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924177 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-run-ovn-kubernetes\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.925642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924182 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwhr6\" (UniqueName: \"kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6\") pod \"network-check-target-jz68h\" (UID: \"cbe6b915-efe6-4616-b39e-cc1066ebd8e4\") " pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:23.925642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924215 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt9dj\" (UniqueName: \"kubernetes.io/projected/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-kube-api-access-rt9dj\") pod \"network-metrics-daemon-vpn5b\" (UID: \"738b8868-dacb-4206-bb1f-cbc6fdb57ecd\") " pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:23.925642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924235 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-host-var-lib-cni-bin\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.925642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924256 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aab50e53-03f9-49ee-8382-c16c7ee45cf2-tmp\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:23.925642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924276 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret\") pod \"global-pull-secret-syncer-kj2dr\" (UID: \"8dcab0f9-5abb-4fa7-af28-54be545eb542\") " pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:23.925642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924278 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-host-var-lib-cni-bin\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.925642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924298 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-var-lib-kubelet\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:23.925642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924320 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6411f69-741b-44f2-ae0f-7babdd39a621-cnibin\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:23.925642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924343 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.925642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924362 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.925642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924379 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.925642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924382 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-device-dir\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.925642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924407 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-etc-selinux\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.925642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924424 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.925642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924430 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-run-systemd\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.926444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924448 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-run-systemd\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.926444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924453 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-etc-openvswitch\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.926444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924433 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-device-dir\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.926444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924472 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-run-openvswitch\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.926444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924493 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-sys\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:23.926444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924496 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-etc-selinux\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.926444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924491 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-etc-openvswitch\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.926444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924512 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-run-openvswitch\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.926444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924522 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/376f8f23-3eb3-46bc-aaec-7bd8007fe531-ovn-node-metrics-cert\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.926444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924545 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8dcab0f9-5abb-4fa7-af28-54be545eb542-kubelet-config\") pod \"global-pull-secret-syncer-kj2dr\" (UID: \"8dcab0f9-5abb-4fa7-af28-54be545eb542\") " pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:23.926444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924562 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-socket-dir\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.926444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924577 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55262c75-f4d2-4f27-a369-e599c35d58a5-host-slash\") pod \"iptables-alerter-f9mq2\" (UID: \"55262c75-f4d2-4f27-a369-e599c35d58a5\") " pod="openshift-network-operator/iptables-alerter-f9mq2" Jul 13 17:05:23.926444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924606 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55262c75-f4d2-4f27-a369-e599c35d58a5-host-slash\") pod \"iptables-alerter-f9mq2\" (UID: \"55262c75-f4d2-4f27-a369-e599c35d58a5\") " pod="openshift-network-operator/iptables-alerter-f9mq2" Jul 13 17:05:23.926444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924624 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-system-cni-dir\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.926444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924640 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-host-run-multus-certs\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.926444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924659 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-host\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:23.926444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924681 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-socket-dir\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.927264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924689 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6411f69-741b-44f2-ae0f-7babdd39a621-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:23.927264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924689 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-system-cni-dir\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.927264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924683 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-host-run-multus-certs\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.927264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924771 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-run-ovn\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.927264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924804 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-cnibin\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.927264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924827 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-kubernetes\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:23.927264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924840 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-run-ovn\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.927264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924854 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e6411f69-741b-44f2-ae0f-7babdd39a621-os-release\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:23.927264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924859 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c891937a-13fe-41d6-b824-2d569278dedb-cnibin\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.927264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924881 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6411f69-741b-44f2-ae0f-7babdd39a621-cni-binary-copy\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:23.927264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924911 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-var-lib-openvswitch\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.927264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924931 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-cni-bin\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.927264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924966 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-var-lib-openvswitch\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.927264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.924989 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/376f8f23-3eb3-46bc-aaec-7bd8007fe531-host-cni-bin\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.927264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.926514 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/71c06287-e098-4864-9a00-fc294a063aa0-agent-certs\") pod \"konnectivity-agent-c8xq8\" (UID: \"71c06287-e098-4864-9a00-fc294a063aa0\") " pod="kube-system/konnectivity-agent-c8xq8" Jul 13 17:05:23.927264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.926861 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/376f8f23-3eb3-46bc-aaec-7bd8007fe531-ovn-node-metrics-cert\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.930095 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.930075 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlpwf\" (UniqueName: \"kubernetes.io/projected/c891937a-13fe-41d6-b824-2d569278dedb-kube-api-access-tlpwf\") pod \"multus-4fwz6\" (UID: \"c891937a-13fe-41d6-b824-2d569278dedb\") " pod="openshift-multus/multus-4fwz6" Jul 13 17:05:23.930932 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.930910 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slj4v\" (UniqueName: \"kubernetes.io/projected/c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d-kube-api-access-slj4v\") pod \"aws-ebs-csi-driver-node-85m7q\" (UID: \"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:23.931747 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.931697 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sf7s\" (UniqueName: \"kubernetes.io/projected/55262c75-f4d2-4f27-a369-e599c35d58a5-kube-api-access-6sf7s\") pod \"iptables-alerter-f9mq2\" (UID: \"55262c75-f4d2-4f27-a369-e599c35d58a5\") " pod="openshift-network-operator/iptables-alerter-f9mq2" Jul 13 17:05:23.932387 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.932364 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzdkb\" (UniqueName: \"kubernetes.io/projected/376f8f23-3eb3-46bc-aaec-7bd8007fe531-kube-api-access-fzdkb\") pod \"ovnkube-node-wmxtg\" (UID: \"376f8f23-3eb3-46bc-aaec-7bd8007fe531\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:23.932643 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.932626 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29b62\" (UniqueName: \"kubernetes.io/projected/ab1216cc-4528-474c-a329-daefce87939c-kube-api-access-29b62\") pod \"node-ca-vq8xq\" (UID: \"ab1216cc-4528-474c-a329-daefce87939c\") " pod="openshift-image-registry/node-ca-vq8xq" Jul 13 17:05:23.933541 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.933515 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt9dj\" (UniqueName: \"kubernetes.io/projected/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-kube-api-access-rt9dj\") pod \"network-metrics-daemon-vpn5b\" (UID: \"738b8868-dacb-4206-bb1f-cbc6fdb57ecd\") " pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:23.970774 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:23.970712 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cdaf1109a9a8f4611e430e48feed9fc.slice/crio-5161b14393db88311d04ac7cab18e34ff88105bb81fa0f663557072706a79dfd WatchSource:0}: Error finding container 5161b14393db88311d04ac7cab18e34ff88105bb81fa0f663557072706a79dfd: Status 404 returned error can't find the container with id 5161b14393db88311d04ac7cab18e34ff88105bb81fa0f663557072706a79dfd Jul 13 17:05:23.971377 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:23.971355 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabc7e90015ad279ad8efd2a082fbdc6c.slice/crio-921fbedec5d8fa806d23ec9874867006c1835d4b579c3573e51f6aaf13795b19 WatchSource:0}: Error finding container 921fbedec5d8fa806d23ec9874867006c1835d4b579c3573e51f6aaf13795b19: Status 404 returned error can't find the container with id 921fbedec5d8fa806d23ec9874867006c1835d4b579c3573e51f6aaf13795b19 Jul 13 17:05:23.976155 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:23.976138 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 17:05:24.025674 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.025591 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aab50e53-03f9-49ee-8382-c16c7ee45cf2-tmp\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.025674 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.025634 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret\") pod \"global-pull-secret-syncer-kj2dr\" (UID: \"8dcab0f9-5abb-4fa7-af28-54be545eb542\") " pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:24.025872 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:24.025750 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:24.025872 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.025771 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-var-lib-kubelet\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.025872 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:24.025807 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret podName:8dcab0f9-5abb-4fa7-af28-54be545eb542 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:24.525788255 +0000 UTC m=+2.188563441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret") pod "global-pull-secret-syncer-kj2dr" (UID: "8dcab0f9-5abb-4fa7-af28-54be545eb542") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:24.025872 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.025834 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-var-lib-kubelet\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.025872 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.025835 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6411f69-741b-44f2-ae0f-7babdd39a621-cnibin\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:24.026055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.025871 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6411f69-741b-44f2-ae0f-7babdd39a621-cnibin\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:24.026055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.025886 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-sys\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.026055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.025931 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-sys\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.026055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.025934 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8dcab0f9-5abb-4fa7-af28-54be545eb542-kubelet-config\") pod \"global-pull-secret-syncer-kj2dr\" (UID: \"8dcab0f9-5abb-4fa7-af28-54be545eb542\") " pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:24.026055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.025962 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-host\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.026055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.025987 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6411f69-741b-44f2-ae0f-7babdd39a621-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:24.026055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026001 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8dcab0f9-5abb-4fa7-af28-54be545eb542-kubelet-config\") pod \"global-pull-secret-syncer-kj2dr\" (UID: \"8dcab0f9-5abb-4fa7-af28-54be545eb542\") " pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:24.026055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026016 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-kubernetes\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.026055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026024 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-host\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.026055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026040 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e6411f69-741b-44f2-ae0f-7babdd39a621-os-release\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:24.026431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026079 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-kubernetes\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.026431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026097 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e6411f69-741b-44f2-ae0f-7babdd39a621-os-release\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:24.026431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026108 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6411f69-741b-44f2-ae0f-7babdd39a621-cni-binary-copy\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:24.026431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026139 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-modprobe-d\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.026431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026167 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d-hosts-file\") pod \"node-resolver-bm8v7\" (UID: \"33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d\") " pod="openshift-dns/node-resolver-bm8v7" Jul 13 17:05:24.026431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026171 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6411f69-741b-44f2-ae0f-7babdd39a621-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:24.026431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026191 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxllm\" (UniqueName: \"kubernetes.io/projected/e6411f69-741b-44f2-ae0f-7babdd39a621-kube-api-access-vxllm\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:24.026431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026223 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e6411f69-741b-44f2-ae0f-7babdd39a621-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:24.026431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026229 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d-hosts-file\") pod \"node-resolver-bm8v7\" (UID: \"33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d\") " pod="openshift-dns/node-resolver-bm8v7" Jul 13 17:05:24.026431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026257 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-tuned\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.026431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026281 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8dcab0f9-5abb-4fa7-af28-54be545eb542-dbus\") pod \"global-pull-secret-syncer-kj2dr\" (UID: \"8dcab0f9-5abb-4fa7-af28-54be545eb542\") " pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:24.026431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026292 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-modprobe-d\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.026431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026308 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-systemd\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.026431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026345 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-systemd\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.026431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026347 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-sysctl-conf\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.026431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026380 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvjs5\" (UniqueName: \"kubernetes.io/projected/aab50e53-03f9-49ee-8382-c16c7ee45cf2-kube-api-access-vvjs5\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.026431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026421 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d-tmp-dir\") pod \"node-resolver-bm8v7\" (UID: \"33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d\") " pod="openshift-dns/node-resolver-bm8v7" Jul 13 17:05:24.027197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026445 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-sysconfig\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.027197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026456 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-sysctl-conf\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.027197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026470 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-run\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.027197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026501 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qz9w\" (UniqueName: \"kubernetes.io/projected/33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d-kube-api-access-2qz9w\") pod \"node-resolver-bm8v7\" (UID: \"33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d\") " pod="openshift-dns/node-resolver-bm8v7" Jul 13 17:05:24.027197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026527 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-sysctl-d\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.027197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026549 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-lib-modules\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.027197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026572 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6411f69-741b-44f2-ae0f-7babdd39a621-system-cni-dir\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:24.027197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026597 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e6411f69-741b-44f2-ae0f-7babdd39a621-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:24.027197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026768 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8dcab0f9-5abb-4fa7-af28-54be545eb542-dbus\") pod \"global-pull-secret-syncer-kj2dr\" (UID: \"8dcab0f9-5abb-4fa7-af28-54be545eb542\") " pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:24.027197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026783 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d-tmp-dir\") pod \"node-resolver-bm8v7\" (UID: \"33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d\") " pod="openshift-dns/node-resolver-bm8v7" Jul 13 17:05:24.027197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026836 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e6411f69-741b-44f2-ae0f-7babdd39a621-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:24.027197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026841 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-sysconfig\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.027197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026868 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-lib-modules\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.027197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026883 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6411f69-741b-44f2-ae0f-7babdd39a621-system-cni-dir\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:24.027197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026894 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-run\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.027197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026914 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwhr6\" (UniqueName: \"kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6\") pod \"network-check-target-jz68h\" (UID: \"cbe6b915-efe6-4616-b39e-cc1066ebd8e4\") " pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:24.027197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.026925 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-sysctl-d\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.027705 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.027042 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e6411f69-741b-44f2-ae0f-7babdd39a621-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:24.027705 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.027091 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6411f69-741b-44f2-ae0f-7babdd39a621-cni-binary-copy\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:24.028125 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.028099 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aab50e53-03f9-49ee-8382-c16c7ee45cf2-tmp\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.028671 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.028655 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aab50e53-03f9-49ee-8382-c16c7ee45cf2-etc-tuned\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.034060 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:24.034040 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:05:24.034060 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:24.034062 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:05:24.034212 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:24.034075 2564 projected.go:196] Error preparing data for projected volume kube-api-access-mwhr6 for pod openshift-network-diagnostics/network-check-target-jz68h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:24.034212 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:24.034145 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6 podName:cbe6b915-efe6-4616-b39e-cc1066ebd8e4 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:24.534126366 +0000 UTC m=+2.196901551 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mwhr6" (UniqueName: "kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6") pod "network-check-target-jz68h" (UID: "cbe6b915-efe6-4616-b39e-cc1066ebd8e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:24.035477 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.035450 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxllm\" (UniqueName: \"kubernetes.io/projected/e6411f69-741b-44f2-ae0f-7babdd39a621-kube-api-access-vxllm\") pod \"multus-additional-cni-plugins-wc8ng\" (UID: \"e6411f69-741b-44f2-ae0f-7babdd39a621\") " pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:24.036746 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.036706 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qz9w\" (UniqueName: \"kubernetes.io/projected/33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d-kube-api-access-2qz9w\") pod \"node-resolver-bm8v7\" (UID: \"33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d\") " pod="openshift-dns/node-resolver-bm8v7" Jul 13 17:05:24.037188 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.037167 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvjs5\" (UniqueName: \"kubernetes.io/projected/aab50e53-03f9-49ee-8382-c16c7ee45cf2-kube-api-access-vvjs5\") pod \"tuned-frz8b\" (UID: \"aab50e53-03f9-49ee-8382-c16c7ee45cf2\") " pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.136353 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.136321 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c8xq8" Jul 13 17:05:24.144444 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:24.144420 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71c06287_e098_4864_9a00_fc294a063aa0.slice/crio-7ef1cf4555936e0bcbcc596e489e9aa70cb114e214ff08089a92661cbc1e40ad WatchSource:0}: Error finding container 7ef1cf4555936e0bcbcc596e489e9aa70cb114e214ff08089a92661cbc1e40ad: Status 404 returned error can't find the container with id 7ef1cf4555936e0bcbcc596e489e9aa70cb114e214ff08089a92661cbc1e40ad Jul 13 17:05:24.156157 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.156136 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vq8xq" Jul 13 17:05:24.163327 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:24.163305 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab1216cc_4528_474c_a329_daefce87939c.slice/crio-a3c27f9402345106c040351e080efe35e78fe6635aab876926fb495699ebfbd9 WatchSource:0}: Error finding container a3c27f9402345106c040351e080efe35e78fe6635aab876926fb495699ebfbd9: Status 404 returned error can't find the container with id a3c27f9402345106c040351e080efe35e78fe6635aab876926fb495699ebfbd9 Jul 13 17:05:24.171113 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.171094 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" Jul 13 17:05:24.178622 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:24.178600 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ce2d14_d22d_4ef5_b313_0b8b31fdb27d.slice/crio-177adc70f229d0e86b09987c0c751080c913393c5567f3c69c6d39a2019fd62b WatchSource:0}: Error finding container 177adc70f229d0e86b09987c0c751080c913393c5567f3c69c6d39a2019fd62b: Status 404 returned error can't find the container with id 177adc70f229d0e86b09987c0c751080c913393c5567f3c69c6d39a2019fd62b Jul 13 17:05:24.193644 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.193613 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f9mq2" Jul 13 17:05:24.200376 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.200350 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:24.200571 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:24.200552 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55262c75_f4d2_4f27_a369_e599c35d58a5.slice/crio-f16d93d1b335cbdf933e96f6b7c8615772f7432ac16ab41958e102490352c685 WatchSource:0}: Error finding container f16d93d1b335cbdf933e96f6b7c8615772f7432ac16ab41958e102490352c685: Status 404 returned error can't find the container with id f16d93d1b335cbdf933e96f6b7c8615772f7432ac16ab41958e102490352c685 Jul 13 17:05:24.207303 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:24.207275 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod376f8f23_3eb3_46bc_aaec_7bd8007fe531.slice/crio-e22ca41c22c2f44b3cbe597325a1906847318257a96f74880b99b7403cd31ce4 WatchSource:0}: Error finding container e22ca41c22c2f44b3cbe597325a1906847318257a96f74880b99b7403cd31ce4: Status 404 returned error can't find the container with id e22ca41c22c2f44b3cbe597325a1906847318257a96f74880b99b7403cd31ce4 Jul 13 17:05:24.213005 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.212982 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4fwz6" Jul 13 17:05:24.220069 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:24.219862 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc891937a_13fe_41d6_b824_2d569278dedb.slice/crio-eee5a4b50fb5f01f6069106bc5ac4aa129e925d6929f99b580266689b458c4d4 WatchSource:0}: Error finding container eee5a4b50fb5f01f6069106bc5ac4aa129e925d6929f99b580266689b458c4d4: Status 404 returned error can't find the container with id eee5a4b50fb5f01f6069106bc5ac4aa129e925d6929f99b580266689b458c4d4 Jul 13 17:05:24.227869 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.227855 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bm8v7" Jul 13 17:05:24.234839 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:24.234815 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33e43fe0_fc83_41c0_8a5c_9cdf168c0d9d.slice/crio-f60b890ab74331096fac700b648b2f33bbe340c8a243b43f3fa169c71124f52d WatchSource:0}: Error finding container f60b890ab74331096fac700b648b2f33bbe340c8a243b43f3fa169c71124f52d: Status 404 returned error can't find the container with id f60b890ab74331096fac700b648b2f33bbe340c8a243b43f3fa169c71124f52d Jul 13 17:05:24.260580 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.260557 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-frz8b" Jul 13 17:05:24.266039 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.266015 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wc8ng" Jul 13 17:05:24.267876 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:24.267853 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaab50e53_03f9_49ee_8382_c16c7ee45cf2.slice/crio-02e8266f3a13c41507c8df4ff167d6243d557fd69d6793e8cace8fca1e3b21c0 WatchSource:0}: Error finding container 02e8266f3a13c41507c8df4ff167d6243d557fd69d6793e8cace8fca1e3b21c0: Status 404 returned error can't find the container with id 02e8266f3a13c41507c8df4ff167d6243d557fd69d6793e8cace8fca1e3b21c0 Jul 13 17:05:24.430265 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.430236 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs\") pod \"network-metrics-daemon-vpn5b\" (UID: \"738b8868-dacb-4206-bb1f-cbc6fdb57ecd\") " pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:24.430406 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:24.430368 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:24.430463 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:24.430427 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs podName:738b8868-dacb-4206-bb1f-cbc6fdb57ecd nodeName:}" failed. No retries permitted until 2026-07-13 17:05:25.430411921 +0000 UTC m=+3.093187098 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs") pod "network-metrics-daemon-vpn5b" (UID: "738b8868-dacb-4206-bb1f-cbc6fdb57ecd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:24.531222 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.531140 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret\") pod \"global-pull-secret-syncer-kj2dr\" (UID: \"8dcab0f9-5abb-4fa7-af28-54be545eb542\") " pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:24.531364 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:24.531300 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:24.531402 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:24.531376 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret podName:8dcab0f9-5abb-4fa7-af28-54be545eb542 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:25.531355781 +0000 UTC m=+3.194130958 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret") pod "global-pull-secret-syncer-kj2dr" (UID: "8dcab0f9-5abb-4fa7-af28-54be545eb542") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:24.632219 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.632181 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwhr6\" (UniqueName: \"kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6\") pod \"network-check-target-jz68h\" (UID: \"cbe6b915-efe6-4616-b39e-cc1066ebd8e4\") " pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:24.632377 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:24.632358 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:05:24.632451 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:24.632384 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:05:24.632451 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:24.632396 2564 projected.go:196] Error preparing data for projected volume kube-api-access-mwhr6 for pod openshift-network-diagnostics/network-check-target-jz68h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:24.632547 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:24.632453 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6 podName:cbe6b915-efe6-4616-b39e-cc1066ebd8e4 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:25.632434025 +0000 UTC m=+3.295209193 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mwhr6" (UniqueName: "kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6") pod "network-check-target-jz68h" (UID: "cbe6b915-efe6-4616-b39e-cc1066ebd8e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:24.821085 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.821014 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 17:05:24.828983 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.828956 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 17:05:24.867828 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.867769 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 17:00:23 +0000 UTC" deadline="2028-01-14 06:59:54.647518857 +0000 UTC" Jul 13 17:05:24.867828 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.867824 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13189h54m29.779698231s" Jul 13 17:05:24.920900 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.920872 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 17:05:24.963023 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.962976 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc8ng" event={"ID":"e6411f69-741b-44f2-ae0f-7babdd39a621","Type":"ContainerStarted","Data":"e34b1a9c2691b926a6829b742f181c1183205439c28fb93612c1f0a0596f607a"} Jul 13 17:05:24.966241 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.966211 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-frz8b" event={"ID":"aab50e53-03f9-49ee-8382-c16c7ee45cf2","Type":"ContainerStarted","Data":"02e8266f3a13c41507c8df4ff167d6243d557fd69d6793e8cace8fca1e3b21c0"} Jul 13 17:05:24.967312 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.967287 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bm8v7" event={"ID":"33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d","Type":"ContainerStarted","Data":"f60b890ab74331096fac700b648b2f33bbe340c8a243b43f3fa169c71124f52d"} Jul 13 17:05:24.969279 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.969230 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4fwz6" event={"ID":"c891937a-13fe-41d6-b824-2d569278dedb","Type":"ContainerStarted","Data":"eee5a4b50fb5f01f6069106bc5ac4aa129e925d6929f99b580266689b458c4d4"} Jul 13 17:05:24.972412 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.972369 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" event={"ID":"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d","Type":"ContainerStarted","Data":"177adc70f229d0e86b09987c0c751080c913393c5567f3c69c6d39a2019fd62b"} Jul 13 17:05:24.975826 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.975801 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c8xq8" event={"ID":"71c06287-e098-4864-9a00-fc294a063aa0","Type":"ContainerStarted","Data":"7ef1cf4555936e0bcbcc596e489e9aa70cb114e214ff08089a92661cbc1e40ad"} Jul 13 17:05:24.977757 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.977708 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal" event={"ID":"0cdaf1109a9a8f4611e430e48feed9fc","Type":"ContainerStarted","Data":"5161b14393db88311d04ac7cab18e34ff88105bb81fa0f663557072706a79dfd"} Jul 13 17:05:24.979216 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.979193 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-245.ec2.internal" event={"ID":"abc7e90015ad279ad8efd2a082fbdc6c","Type":"ContainerStarted","Data":"921fbedec5d8fa806d23ec9874867006c1835d4b579c3573e51f6aaf13795b19"} Jul 13 17:05:24.980649 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.980623 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" event={"ID":"376f8f23-3eb3-46bc-aaec-7bd8007fe531","Type":"ContainerStarted","Data":"e22ca41c22c2f44b3cbe597325a1906847318257a96f74880b99b7403cd31ce4"} Jul 13 17:05:24.983049 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.983023 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f9mq2" event={"ID":"55262c75-f4d2-4f27-a369-e599c35d58a5","Type":"ContainerStarted","Data":"f16d93d1b335cbdf933e96f6b7c8615772f7432ac16ab41958e102490352c685"} Jul 13 17:05:24.984783 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:24.984720 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vq8xq" event={"ID":"ab1216cc-4528-474c-a329-daefce87939c","Type":"ContainerStarted","Data":"a3c27f9402345106c040351e080efe35e78fe6635aab876926fb495699ebfbd9"} Jul 13 17:05:25.439246 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:25.439205 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs\") pod \"network-metrics-daemon-vpn5b\" (UID: \"738b8868-dacb-4206-bb1f-cbc6fdb57ecd\") " pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:25.439419 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:25.439357 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:25.439460 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:25.439422 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs podName:738b8868-dacb-4206-bb1f-cbc6fdb57ecd nodeName:}" failed. No retries permitted until 2026-07-13 17:05:27.4394022 +0000 UTC m=+5.102177380 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs") pod "network-metrics-daemon-vpn5b" (UID: "738b8868-dacb-4206-bb1f-cbc6fdb57ecd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:25.540649 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:25.540603 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret\") pod \"global-pull-secret-syncer-kj2dr\" (UID: \"8dcab0f9-5abb-4fa7-af28-54be545eb542\") " pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:25.540871 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:25.540819 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:25.540929 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:25.540890 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret podName:8dcab0f9-5abb-4fa7-af28-54be545eb542 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:27.540869364 +0000 UTC m=+5.203644532 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret") pod "global-pull-secret-syncer-kj2dr" (UID: "8dcab0f9-5abb-4fa7-af28-54be545eb542") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:25.641762 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:25.641704 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwhr6\" (UniqueName: \"kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6\") pod \"network-check-target-jz68h\" (UID: \"cbe6b915-efe6-4616-b39e-cc1066ebd8e4\") " pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:25.641936 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:25.641917 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:05:25.641994 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:25.641938 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:05:25.641994 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:25.641951 2564 projected.go:196] Error preparing data for projected volume kube-api-access-mwhr6 for pod openshift-network-diagnostics/network-check-target-jz68h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:25.642096 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:25.642007 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6 podName:cbe6b915-efe6-4616-b39e-cc1066ebd8e4 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:27.641989797 +0000 UTC m=+5.304764984 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mwhr6" (UniqueName: "kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6") pod "network-check-target-jz68h" (UID: "cbe6b915-efe6-4616-b39e-cc1066ebd8e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:25.868112 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:25.868001 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 17:00:23 +0000 UTC" deadline="2028-03-30 20:00:56.28570274 +0000 UTC" Jul 13 17:05:25.868112 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:25.868046 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15026h55m30.417661701s" Jul 13 17:05:25.955417 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:25.955330 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:25.955600 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:25.955465 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj2dr" podUID="8dcab0f9-5abb-4fa7-af28-54be545eb542" Jul 13 17:05:25.955910 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:25.955888 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:25.956024 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:25.955999 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpn5b" podUID="738b8868-dacb-4206-bb1f-cbc6fdb57ecd" Jul 13 17:05:25.956173 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:25.956058 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:25.956173 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:25.956123 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz68h" podUID="cbe6b915-efe6-4616-b39e-cc1066ebd8e4" Jul 13 17:05:27.456336 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:27.456227 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs\") pod \"network-metrics-daemon-vpn5b\" (UID: \"738b8868-dacb-4206-bb1f-cbc6fdb57ecd\") " pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:27.456814 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:27.456356 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:27.456814 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:27.456429 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs podName:738b8868-dacb-4206-bb1f-cbc6fdb57ecd nodeName:}" failed. No retries permitted until 2026-07-13 17:05:31.456409708 +0000 UTC m=+9.119184888 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs") pod "network-metrics-daemon-vpn5b" (UID: "738b8868-dacb-4206-bb1f-cbc6fdb57ecd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:27.557452 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:27.557420 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret\") pod \"global-pull-secret-syncer-kj2dr\" (UID: \"8dcab0f9-5abb-4fa7-af28-54be545eb542\") " pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:27.557608 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:27.557537 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:27.557608 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:27.557596 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret podName:8dcab0f9-5abb-4fa7-af28-54be545eb542 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:31.557580414 +0000 UTC m=+9.220355581 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret") pod "global-pull-secret-syncer-kj2dr" (UID: "8dcab0f9-5abb-4fa7-af28-54be545eb542") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:27.658155 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:27.658009 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwhr6\" (UniqueName: \"kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6\") pod \"network-check-target-jz68h\" (UID: \"cbe6b915-efe6-4616-b39e-cc1066ebd8e4\") " pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:27.658342 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:27.658173 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:05:27.658342 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:27.658193 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:05:27.658342 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:27.658206 2564 projected.go:196] Error preparing data for projected volume kube-api-access-mwhr6 for pod openshift-network-diagnostics/network-check-target-jz68h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:27.658342 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:27.658258 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6 podName:cbe6b915-efe6-4616-b39e-cc1066ebd8e4 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:31.658244685 +0000 UTC m=+9.321019851 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mwhr6" (UniqueName: "kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6") pod "network-check-target-jz68h" (UID: "cbe6b915-efe6-4616-b39e-cc1066ebd8e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:27.955308 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:27.955226 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:27.955464 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:27.955359 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj2dr" podUID="8dcab0f9-5abb-4fa7-af28-54be545eb542" Jul 13 17:05:27.955763 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:27.955743 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:27.955867 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:27.955847 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpn5b" podUID="738b8868-dacb-4206-bb1f-cbc6fdb57ecd" Jul 13 17:05:27.955914 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:27.955904 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:27.955969 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:27.955954 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz68h" podUID="cbe6b915-efe6-4616-b39e-cc1066ebd8e4" Jul 13 17:05:29.955281 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:29.954808 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:29.955281 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:29.954847 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:29.955281 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:29.954926 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz68h" podUID="cbe6b915-efe6-4616-b39e-cc1066ebd8e4" Jul 13 17:05:29.955281 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:29.955010 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:29.955281 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:29.955142 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpn5b" podUID="738b8868-dacb-4206-bb1f-cbc6fdb57ecd" Jul 13 17:05:29.955281 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:29.955242 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj2dr" podUID="8dcab0f9-5abb-4fa7-af28-54be545eb542" Jul 13 17:05:31.493492 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:31.493423 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs\") pod \"network-metrics-daemon-vpn5b\" (UID: \"738b8868-dacb-4206-bb1f-cbc6fdb57ecd\") " pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:31.493975 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:31.493573 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:31.493975 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:31.493640 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs podName:738b8868-dacb-4206-bb1f-cbc6fdb57ecd nodeName:}" failed. No retries permitted until 2026-07-13 17:05:39.493622924 +0000 UTC m=+17.156398102 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs") pod "network-metrics-daemon-vpn5b" (UID: "738b8868-dacb-4206-bb1f-cbc6fdb57ecd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:31.594464 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:31.594419 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret\") pod \"global-pull-secret-syncer-kj2dr\" (UID: \"8dcab0f9-5abb-4fa7-af28-54be545eb542\") " pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:31.594655 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:31.594576 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:31.594655 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:31.594639 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret podName:8dcab0f9-5abb-4fa7-af28-54be545eb542 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:39.594621423 +0000 UTC m=+17.257396616 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret") pod "global-pull-secret-syncer-kj2dr" (UID: "8dcab0f9-5abb-4fa7-af28-54be545eb542") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:31.695551 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:31.695468 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwhr6\" (UniqueName: \"kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6\") pod \"network-check-target-jz68h\" (UID: \"cbe6b915-efe6-4616-b39e-cc1066ebd8e4\") " pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:31.695710 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:31.695641 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:05:31.695710 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:31.695665 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:05:31.695710 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:31.695678 2564 projected.go:196] Error preparing data for projected volume kube-api-access-mwhr6 for pod openshift-network-diagnostics/network-check-target-jz68h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:31.695914 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:31.695754 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6 podName:cbe6b915-efe6-4616-b39e-cc1066ebd8e4 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:39.695718319 +0000 UTC m=+17.358493499 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mwhr6" (UniqueName: "kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6") pod "network-check-target-jz68h" (UID: "cbe6b915-efe6-4616-b39e-cc1066ebd8e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:31.955177 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:31.955084 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:31.955177 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:31.955131 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:31.955420 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:31.955089 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:31.955420 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:31.955227 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz68h" podUID="cbe6b915-efe6-4616-b39e-cc1066ebd8e4" Jul 13 17:05:31.955420 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:31.955320 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpn5b" podUID="738b8868-dacb-4206-bb1f-cbc6fdb57ecd" Jul 13 17:05:31.955420 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:31.955413 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj2dr" podUID="8dcab0f9-5abb-4fa7-af28-54be545eb542" Jul 13 17:05:33.955581 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:33.954926 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:33.955581 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:33.955054 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz68h" podUID="cbe6b915-efe6-4616-b39e-cc1066ebd8e4" Jul 13 17:05:33.955581 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:33.955371 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:33.955581 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:33.955440 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj2dr" podUID="8dcab0f9-5abb-4fa7-af28-54be545eb542" Jul 13 17:05:33.955581 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:33.955475 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:33.955581 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:33.955533 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpn5b" podUID="738b8868-dacb-4206-bb1f-cbc6fdb57ecd" Jul 13 17:05:35.955039 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:35.954998 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:35.955480 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:35.955046 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:35.955480 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:35.955046 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:35.955480 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:35.955125 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz68h" podUID="cbe6b915-efe6-4616-b39e-cc1066ebd8e4" Jul 13 17:05:35.955480 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:35.955251 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj2dr" podUID="8dcab0f9-5abb-4fa7-af28-54be545eb542" Jul 13 17:05:35.955480 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:35.955339 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpn5b" podUID="738b8868-dacb-4206-bb1f-cbc6fdb57ecd" Jul 13 17:05:37.955564 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:37.955524 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:37.955564 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:37.955553 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:37.955564 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:37.955568 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:37.956125 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:37.955654 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpn5b" podUID="738b8868-dacb-4206-bb1f-cbc6fdb57ecd" Jul 13 17:05:37.956125 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:37.955747 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz68h" podUID="cbe6b915-efe6-4616-b39e-cc1066ebd8e4" Jul 13 17:05:37.956125 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:37.955802 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj2dr" podUID="8dcab0f9-5abb-4fa7-af28-54be545eb542" Jul 13 17:05:39.553409 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:39.553362 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs\") pod \"network-metrics-daemon-vpn5b\" (UID: \"738b8868-dacb-4206-bb1f-cbc6fdb57ecd\") " pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:39.553816 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:39.553533 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:39.553816 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:39.553611 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs podName:738b8868-dacb-4206-bb1f-cbc6fdb57ecd nodeName:}" failed. No retries permitted until 2026-07-13 17:05:55.553586794 +0000 UTC m=+33.216361980 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs") pod "network-metrics-daemon-vpn5b" (UID: "738b8868-dacb-4206-bb1f-cbc6fdb57ecd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:39.653955 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:39.653917 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret\") pod \"global-pull-secret-syncer-kj2dr\" (UID: \"8dcab0f9-5abb-4fa7-af28-54be545eb542\") " pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:39.654122 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:39.654063 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:39.654161 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:39.654128 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret podName:8dcab0f9-5abb-4fa7-af28-54be545eb542 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:55.654110954 +0000 UTC m=+33.316886124 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret") pod "global-pull-secret-syncer-kj2dr" (UID: "8dcab0f9-5abb-4fa7-af28-54be545eb542") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:39.754869 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:39.754829 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwhr6\" (UniqueName: \"kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6\") pod \"network-check-target-jz68h\" (UID: \"cbe6b915-efe6-4616-b39e-cc1066ebd8e4\") " pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:39.755063 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:39.755008 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:05:39.755063 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:39.755034 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:05:39.755063 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:39.755047 2564 projected.go:196] Error preparing data for projected volume kube-api-access-mwhr6 for pod openshift-network-diagnostics/network-check-target-jz68h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:39.755209 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:39.755100 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6 podName:cbe6b915-efe6-4616-b39e-cc1066ebd8e4 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:55.755084672 +0000 UTC m=+33.417859851 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mwhr6" (UniqueName: "kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6") pod "network-check-target-jz68h" (UID: "cbe6b915-efe6-4616-b39e-cc1066ebd8e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:39.955336 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:39.955248 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:39.955503 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:39.955368 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:39.955562 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:39.955378 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:39.955562 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:39.955542 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj2dr" podUID="8dcab0f9-5abb-4fa7-af28-54be545eb542" Jul 13 17:05:39.955667 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:39.955377 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz68h" podUID="cbe6b915-efe6-4616-b39e-cc1066ebd8e4" Jul 13 17:05:39.955716 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:39.955641 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpn5b" podUID="738b8868-dacb-4206-bb1f-cbc6fdb57ecd" Jul 13 17:05:41.955756 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:41.955509 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:41.955756 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:41.955567 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:41.955756 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:41.955628 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:41.955756 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:41.955665 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz68h" podUID="cbe6b915-efe6-4616-b39e-cc1066ebd8e4" Jul 13 17:05:41.955756 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:41.955677 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj2dr" podUID="8dcab0f9-5abb-4fa7-af28-54be545eb542" Jul 13 17:05:41.956322 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:41.955775 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpn5b" podUID="738b8868-dacb-4206-bb1f-cbc6fdb57ecd" Jul 13 17:05:43.955368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:43.955328 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:43.955368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:43.955366 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:43.955852 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:43.955366 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:43.955852 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:43.955434 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpn5b" podUID="738b8868-dacb-4206-bb1f-cbc6fdb57ecd" Jul 13 17:05:43.955852 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:43.955553 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz68h" podUID="cbe6b915-efe6-4616-b39e-cc1066ebd8e4" Jul 13 17:05:43.955852 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:43.955650 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj2dr" podUID="8dcab0f9-5abb-4fa7-af28-54be545eb542" Jul 13 17:05:45.027093 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.026657 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4fwz6" event={"ID":"c891937a-13fe-41d6-b824-2d569278dedb","Type":"ContainerStarted","Data":"43c67fad5c640d0882467a8026c86fcb9b55ae0987d476c07c75fff8e90e47ca"} Jul 13 17:05:45.028233 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.028205 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" event={"ID":"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d","Type":"ContainerStarted","Data":"9385dd7a39ccb07bfdc2e4eb69b5bcd1a236a339ba714c3280363bf87896e1c2"} Jul 13 17:05:45.029412 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.029389 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c8xq8" event={"ID":"71c06287-e098-4864-9a00-fc294a063aa0","Type":"ContainerStarted","Data":"e8c2a6624492142cbf438ae6828b18c5bcbed976ecdac69e3b10e5152cada916"} Jul 13 17:05:45.030968 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.030947 2564 generic.go:356] "Generic (PLEG): container finished" podID="0cdaf1109a9a8f4611e430e48feed9fc" containerID="54f3990e73dc888104302e40c6bc72a3cf5b6fbbe5f505024ce33aae9b28ceb3" exitCode=0 Jul 13 17:05:45.031062 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.031016 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal" event={"ID":"0cdaf1109a9a8f4611e430e48feed9fc","Type":"ContainerDied","Data":"54f3990e73dc888104302e40c6bc72a3cf5b6fbbe5f505024ce33aae9b28ceb3"} Jul 13 17:05:45.032474 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.032455 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-245.ec2.internal" event={"ID":"abc7e90015ad279ad8efd2a082fbdc6c","Type":"ContainerStarted","Data":"a90697142812ae5935d576b02a25a102d001d1b9339de80d713afc2507ba7587"} Jul 13 17:05:45.034275 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.034254 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" event={"ID":"376f8f23-3eb3-46bc-aaec-7bd8007fe531","Type":"ContainerStarted","Data":"ba67e94c4a6ae3a77ecc7244136a851a853930fd42f114e8e6829013bbf7a25e"} Jul 13 17:05:45.034359 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.034314 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" event={"ID":"376f8f23-3eb3-46bc-aaec-7bd8007fe531","Type":"ContainerStarted","Data":"b46072d4a2cce97962211619bfa84907c132b5c1009125095328194d4af508f2"} Jul 13 17:05:45.035472 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.035454 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vq8xq" event={"ID":"ab1216cc-4528-474c-a329-daefce87939c","Type":"ContainerStarted","Data":"7798e97c3df60082d6cd3fb40c70587fe838ab4b96ffa24e6527eb849739bdd8"} Jul 13 17:05:45.036921 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.036900 2564 generic.go:356] "Generic (PLEG): container finished" podID="e6411f69-741b-44f2-ae0f-7babdd39a621" containerID="26b80c80565feb7064c4729da5f2a009c96c746594164d8363df26535f717889" exitCode=0 Jul 13 17:05:45.037017 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.036929 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc8ng" event={"ID":"e6411f69-741b-44f2-ae0f-7babdd39a621","Type":"ContainerDied","Data":"26b80c80565feb7064c4729da5f2a009c96c746594164d8363df26535f717889"} Jul 13 17:05:45.038404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.038356 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-frz8b" event={"ID":"aab50e53-03f9-49ee-8382-c16c7ee45cf2","Type":"ContainerStarted","Data":"1e707c3732dddcf5107470355a470d134f6ad9686b60ccd51dde3a2d96c6b2fd"} Jul 13 17:05:45.039760 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.039704 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bm8v7" event={"ID":"33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d","Type":"ContainerStarted","Data":"596bf90dae97d95fa21640f0b83c1317be742484191929521c8183153dbda7d6"} Jul 13 17:05:45.045239 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.045201 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4fwz6" podStartSLOduration=2.969228438 podStartE2EDuration="23.045188967s" totalImagesPullingTime="20.075960529s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:45.044326285 +0000 UTC m=+22.707101475" watchObservedRunningTime="2026-07-13 17:05:45.045188967 +0000 UTC m=+22.707964155" Jul 13 17:05:45.058651 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.058606 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-245.ec2.internal" podStartSLOduration=22.058593662 podStartE2EDuration="22.058593662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:45.058291383 +0000 UTC m=+22.721066571" watchObservedRunningTime="2026-07-13 17:05:45.058593662 +0000 UTC m=+22.721369160" Jul 13 17:05:45.091343 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.091160 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bm8v7" podStartSLOduration=2.426602872 podStartE2EDuration="22.09114097s" totalImagesPullingTime="19.664538098s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:45.090698328 +0000 UTC m=+22.753473515" watchObservedRunningTime="2026-07-13 17:05:45.09114097 +0000 UTC m=+22.753916159" Jul 13 17:05:45.091343 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.091266 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vq8xq" podStartSLOduration=3.013450674 podStartE2EDuration="23.09125778s" totalImagesPullingTime="20.077807106s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:45.073870676 +0000 UTC m=+22.736645863" watchObservedRunningTime="2026-07-13 17:05:45.09125778 +0000 UTC m=+22.754032970" Jul 13 17:05:45.112102 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.112050 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-frz8b" podStartSLOduration=2.11842699 podStartE2EDuration="22.112032306s" totalImagesPullingTime="19.993605316s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:45.111997172 +0000 UTC m=+22.774772360" watchObservedRunningTime="2026-07-13 17:05:45.112032306 +0000 UTC m=+22.774807494" Jul 13 17:05:45.155194 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.155142 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-c8xq8" podStartSLOduration=3.400192133 podStartE2EDuration="23.155126698s" totalImagesPullingTime="19.754934565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:45.154804844 +0000 UTC m=+22.817580029" watchObservedRunningTime="2026-07-13 17:05:45.155126698 +0000 UTC m=+22.817901909" Jul 13 17:05:45.265625 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.265599 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:45.730322 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.730131 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:45.787981 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.787946 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:45.955260 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.955231 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:45.955436 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.955319 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:45.955436 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:45.955323 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz68h" podUID="cbe6b915-efe6-4616-b39e-cc1066ebd8e4" Jul 13 17:05:45.955436 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:45.955408 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj2dr" podUID="8dcab0f9-5abb-4fa7-af28-54be545eb542" Jul 13 17:05:45.955436 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:45.955328 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:45.955575 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:45.955490 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpn5b" podUID="738b8868-dacb-4206-bb1f-cbc6fdb57ecd" Jul 13 17:05:46.043673 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:46.043637 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal" event={"ID":"0cdaf1109a9a8f4611e430e48feed9fc","Type":"ContainerStarted","Data":"98759a61e06d0edf8e4e529943447bd20710f76d691b908a40877351897b29ba"} Jul 13 17:05:46.046617 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:46.046592 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" event={"ID":"376f8f23-3eb3-46bc-aaec-7bd8007fe531","Type":"ContainerStarted","Data":"e93719e0752c1cd32acbf006d5430055bace106686a4b279eea293f8ebd87d72"} Jul 13 17:05:46.046784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:46.046654 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" event={"ID":"376f8f23-3eb3-46bc-aaec-7bd8007fe531","Type":"ContainerStarted","Data":"967884a100b4fb6006fbfdd279faefdcfb70da01ea5bfe12edd2506bd7c77a1a"} Jul 13 17:05:46.046784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:46.046694 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" event={"ID":"376f8f23-3eb3-46bc-aaec-7bd8007fe531","Type":"ContainerStarted","Data":"7df46d1d19715175a79970f963a51988137ace05a951cc5822d3266e340b4454"} Jul 13 17:05:46.046784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:46.046755 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" event={"ID":"376f8f23-3eb3-46bc-aaec-7bd8007fe531","Type":"ContainerStarted","Data":"171a7637e9a662cefe0c8159301777f4cc4dd5f78def31d27e50ce3bb5a9b3fd"} Jul 13 17:05:46.048508 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:46.048483 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f9mq2" event={"ID":"55262c75-f4d2-4f27-a369-e599c35d58a5","Type":"ContainerStarted","Data":"3323563f8fca7d8999586f5c0d8ddbe8d20993289900d2f729c93da091db1815"} Jul 13 17:05:46.059490 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:46.059449 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-245.ec2.internal" podStartSLOduration=23.059435493 podStartE2EDuration="23.059435493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:46.05901117 +0000 UTC m=+23.721786349" watchObservedRunningTime="2026-07-13 17:05:46.059435493 +0000 UTC m=+23.722210683" Jul 13 17:05:46.073831 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:46.073793 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f9mq2" podStartSLOduration=4.013004162 podStartE2EDuration="24.073780022s" totalImagesPullingTime="20.06077586s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:46.073354346 +0000 UTC m=+23.736129534" watchObservedRunningTime="2026-07-13 17:05:46.073780022 +0000 UTC m=+23.736555294" Jul 13 17:05:46.190213 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:46.190156 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 17:05:46.194531 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:46.194505 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:46.294529 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:46.294446 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:46.521638 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:46.521567 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-c8xq8" Jul 13 17:05:46.522187 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:46.522167 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-c8xq8" Jul 13 17:05:46.698379 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:46.698342 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:46.799136 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:46.799086 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:46.887923 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:46.887802 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T17:05:46.190180152Z","UUID":"267b1549-3845-4a40-b1a9-b9d2dd00ed90","Handler":null,"Name":"","Endpoint":""} Jul 13 17:05:46.889634 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:46.889607 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 17:05:46.889787 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:46.889649 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 17:05:47.054996 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:47.054894 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" event={"ID":"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d","Type":"ContainerStarted","Data":"3f9671eee842d070168b0cf600291e2b1e1b26a7c8b13909a96d1495ee03437b"} Jul 13 17:05:47.054996 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:47.054967 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-c8xq8" Jul 13 17:05:47.055667 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:47.055548 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c8xq8" Jul 13 17:05:47.132960 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:47.132925 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:47.954975 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:47.954938 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:47.954975 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:47.954974 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:47.955222 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:47.954980 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:47.955222 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:47.955060 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz68h" podUID="cbe6b915-efe6-4616-b39e-cc1066ebd8e4" Jul 13 17:05:47.955222 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:47.955210 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpn5b" podUID="738b8868-dacb-4206-bb1f-cbc6fdb57ecd" Jul 13 17:05:47.955367 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:47.955298 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj2dr" podUID="8dcab0f9-5abb-4fa7-af28-54be545eb542" Jul 13 17:05:48.059055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:48.058955 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" event={"ID":"c0ce2d14-d22d-4ef5-b313-0b8b31fdb27d","Type":"ContainerStarted","Data":"ccf19909e53429a642b77a2c1b72b49d3d25d2bc0190ac07aeae566fcdd38c6b"} Jul 13 17:05:48.062818 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:48.062790 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" event={"ID":"376f8f23-3eb3-46bc-aaec-7bd8007fe531","Type":"ContainerStarted","Data":"a102e89974c938517470fa9f0e8515139b8893973b86434a0e7089295102b1c2"} Jul 13 17:05:49.955366 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:49.955192 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:49.955816 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:49.955242 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:49.955816 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:49.955450 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpn5b" podUID="738b8868-dacb-4206-bb1f-cbc6fdb57ecd" Jul 13 17:05:49.955816 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:49.955242 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:49.955816 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:49.955527 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz68h" podUID="cbe6b915-efe6-4616-b39e-cc1066ebd8e4" Jul 13 17:05:49.955816 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:49.955591 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj2dr" podUID="8dcab0f9-5abb-4fa7-af28-54be545eb542" Jul 13 17:05:50.071798 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:50.071753 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" event={"ID":"376f8f23-3eb3-46bc-aaec-7bd8007fe531","Type":"ContainerStarted","Data":"8bdd535acaf1ccd9e2ea292eb17771977e5069e54dc5f12691e487c1292b9b09"} Jul 13 17:05:50.071993 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:50.071977 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:50.072061 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:50.072043 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:50.072112 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:50.072092 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:50.088591 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:50.088561 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:50.088896 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:50.088879 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:05:50.104639 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:50.104597 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" podStartSLOduration=7.676391608 podStartE2EDuration="28.104584923s" totalImagesPullingTime="20.428193315s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:50.104458258 +0000 UTC m=+27.767233445" watchObservedRunningTime="2026-07-13 17:05:50.104584923 +0000 UTC m=+27.767360111" Jul 13 17:05:50.105079 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:50.105056 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85m7q" podStartSLOduration=5.594163323 podStartE2EDuration="28.105050195s" totalImagesPullingTime="22.510886872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:48.078891466 +0000 UTC m=+25.741666655" watchObservedRunningTime="2026-07-13 17:05:50.105050195 +0000 UTC m=+27.767825439" Jul 13 17:05:51.077467 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:51.077406 2564 generic.go:356] "Generic (PLEG): container finished" podID="e6411f69-741b-44f2-ae0f-7babdd39a621" containerID="21ad14069c95f7a80047f9ad9dffd9461eb107b5065d0a84f24253c609733b64" exitCode=0 Jul 13 17:05:51.078013 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:51.077497 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc8ng" event={"ID":"e6411f69-741b-44f2-ae0f-7babdd39a621","Type":"ContainerDied","Data":"21ad14069c95f7a80047f9ad9dffd9461eb107b5065d0a84f24253c609733b64"} Jul 13 17:05:51.954789 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:51.954699 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:51.954789 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:51.954760 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:51.954789 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:51.954760 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:51.955150 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:51.954857 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpn5b" podUID="738b8868-dacb-4206-bb1f-cbc6fdb57ecd" Jul 13 17:05:51.955150 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:51.954908 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz68h" podUID="cbe6b915-efe6-4616-b39e-cc1066ebd8e4" Jul 13 17:05:51.955150 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:51.954985 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj2dr" podUID="8dcab0f9-5abb-4fa7-af28-54be545eb542" Jul 13 17:05:52.062910 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:52.062879 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kj2dr"] Jul 13 17:05:52.066141 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:52.066115 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jz68h"] Jul 13 17:05:52.079960 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:52.079937 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vpn5b"] Jul 13 17:05:52.080434 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:52.080177 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:52.080434 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:52.080200 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:52.080434 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:52.080175 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:52.080434 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:52.080295 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj2dr" podUID="8dcab0f9-5abb-4fa7-af28-54be545eb542" Jul 13 17:05:52.080434 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:52.080357 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz68h" podUID="cbe6b915-efe6-4616-b39e-cc1066ebd8e4" Jul 13 17:05:52.080775 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:52.080454 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpn5b" podUID="738b8868-dacb-4206-bb1f-cbc6fdb57ecd" Jul 13 17:05:53.084436 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:53.084402 2564 generic.go:356] "Generic (PLEG): container finished" podID="e6411f69-741b-44f2-ae0f-7babdd39a621" containerID="fa7c2c5597424a46db68acc8c3c48298fe3f3da465c4f192e8e7b07d2953aecb" exitCode=0 Jul 13 17:05:53.084436 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:53.084441 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc8ng" event={"ID":"e6411f69-741b-44f2-ae0f-7babdd39a621","Type":"ContainerDied","Data":"fa7c2c5597424a46db68acc8c3c48298fe3f3da465c4f192e8e7b07d2953aecb"} Jul 13 17:05:53.183243 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:53.183210 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:53.242649 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:53.242620 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:53.640904 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:53.640852 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:53.737618 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:53.737582 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:53.955371 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:53.955340 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:53.955532 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:53.955348 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:53.955532 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:53.955448 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj2dr" podUID="8dcab0f9-5abb-4fa7-af28-54be545eb542" Jul 13 17:05:53.955629 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:53.955542 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz68h" podUID="cbe6b915-efe6-4616-b39e-cc1066ebd8e4" Jul 13 17:05:53.955629 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:53.955348 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:53.955688 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:53.955648 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpn5b" podUID="738b8868-dacb-4206-bb1f-cbc6fdb57ecd" Jul 13 17:05:54.065553 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:54.065518 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:54.165548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:54.165514 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:54.268124 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:54.268093 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:54.364932 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:54.364909 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:54.472341 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:54.472237 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:54.561039 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:54.561007 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:54.773586 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:54.773496 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:54.862203 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:54.862168 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:54.961906 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:54.961868 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:55.091470 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:55.091377 2564 generic.go:356] "Generic (PLEG): container finished" podID="e6411f69-741b-44f2-ae0f-7babdd39a621" containerID="e32db2e01b25d558a69f5b7555f2c4d8defbc3446fea945918b35b79f0876d8f" exitCode=0 Jul 13 17:05:55.091470 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:55.091453 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc8ng" event={"ID":"e6411f69-741b-44f2-ae0f-7babdd39a621","Type":"ContainerDied","Data":"e32db2e01b25d558a69f5b7555f2c4d8defbc3446fea945918b35b79f0876d8f"} Jul 13 17:05:55.194212 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:55.194182 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:55.294347 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:55.294312 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:55.298758 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:55.298744 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bm8v7_33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d/dns-node-resolver/0.log" Jul 13 17:05:55.569838 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:55.569800 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs\") pod \"network-metrics-daemon-vpn5b\" (UID: \"738b8868-dacb-4206-bb1f-cbc6fdb57ecd\") " pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:55.570015 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:55.569961 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:55.570061 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:55.570024 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs podName:738b8868-dacb-4206-bb1f-cbc6fdb57ecd nodeName:}" failed. No retries permitted until 2026-07-13 17:06:27.570008917 +0000 UTC m=+65.232784297 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs") pod "network-metrics-daemon-vpn5b" (UID: "738b8868-dacb-4206-bb1f-cbc6fdb57ecd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:55.670364 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:55.670323 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret\") pod \"global-pull-secret-syncer-kj2dr\" (UID: \"8dcab0f9-5abb-4fa7-af28-54be545eb542\") " pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:55.670503 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:55.670466 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:55.670561 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:55.670528 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret podName:8dcab0f9-5abb-4fa7-af28-54be545eb542 nodeName:}" failed. No retries permitted until 2026-07-13 17:06:27.670509633 +0000 UTC m=+65.333284805 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret") pod "global-pull-secret-syncer-kj2dr" (UID: "8dcab0f9-5abb-4fa7-af28-54be545eb542") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:55.770892 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:55.770850 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwhr6\" (UniqueName: \"kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6\") pod \"network-check-target-jz68h\" (UID: \"cbe6b915-efe6-4616-b39e-cc1066ebd8e4\") " pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:55.771079 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:55.771014 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:05:55.771079 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:55.771039 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:05:55.771079 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:55.771051 2564 projected.go:196] Error preparing data for projected volume kube-api-access-mwhr6 for pod openshift-network-diagnostics/network-check-target-jz68h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:55.771226 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:55.771120 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6 podName:cbe6b915-efe6-4616-b39e-cc1066ebd8e4 nodeName:}" failed. No retries permitted until 2026-07-13 17:06:27.771099833 +0000 UTC m=+65.433875019 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mwhr6" (UniqueName: "kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6") pod "network-check-target-jz68h" (UID: "cbe6b915-efe6-4616-b39e-cc1066ebd8e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:55.955154 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:55.954957 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:55.955338 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:55.954966 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:55.955338 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:55.955238 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj2dr" podUID="8dcab0f9-5abb-4fa7-af28-54be545eb542" Jul 13 17:05:55.955338 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:55.955303 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz68h" podUID="cbe6b915-efe6-4616-b39e-cc1066ebd8e4" Jul 13 17:05:55.955338 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:55.955040 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:55.955473 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:55.955394 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpn5b" podUID="738b8868-dacb-4206-bb1f-cbc6fdb57ecd" Jul 13 17:05:56.503367 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:56.503334 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vq8xq_ab1216cc-4528-474c-a329-daefce87939c/node-ca/0.log" Jul 13 17:05:56.832230 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:56.832154 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:56.900528 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:56.900494 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:57.065605 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.065556 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:57.116147 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.116062 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-245.ec2.internal" event="NodeReady" Jul 13 17:05:57.116314 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.116211 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 17:05:57.155163 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.155133 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d88ff76c8-2m949"] Jul 13 17:05:57.157277 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.157248 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.160249 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.160222 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 13 17:05:57.160390 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.160229 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ctbft\"" Jul 13 17:05:57.160566 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.160541 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 13 17:05:57.160676 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.160595 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 13 17:05:57.170410 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.170389 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 13 17:05:57.177410 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.177386 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-64h5v"] Jul 13 17:05:57.180055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.180032 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d88ff76c8-2m949"] Jul 13 17:05:57.180055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.180067 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qqmx6"] Jul 13 17:05:57.180241 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.180166 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-64h5v" Jul 13 17:05:57.182949 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.182931 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:05:57.183514 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.183357 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 17:05:57.183514 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.183416 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 17:05:57.183696 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.183683 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xtcxn\"" Jul 13 17:05:57.186119 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.186086 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 17:05:57.186119 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.186102 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 17:05:57.186285 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.186102 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 17:05:57.186285 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.186102 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 17:05:57.188229 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.188210 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rmd8z\"" Jul 13 17:05:57.191171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.191151 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jjsrf"] Jul 13 17:05:57.194955 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.194874 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-64h5v"] Jul 13 17:05:57.195072 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.194984 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jjsrf" Jul 13 17:05:57.196586 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.196313 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qqmx6"] Jul 13 17:05:57.200185 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.199991 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 17:05:57.200185 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.200022 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ghws2\"" Jul 13 17:05:57.200185 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.200030 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 17:05:57.200346 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.200224 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 17:05:57.204217 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.204197 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jjsrf"] Jul 13 17:05:57.280599 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.280567 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:05:57.280817 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.280605 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c00d0e29-584d-4428-b879-f98edecd3b85-image-registry-private-configuration\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.280817 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.280634 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c00d0e29-584d-4428-b879-f98edecd3b85-registry-certificates\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.280817 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.280652 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-bound-sa-token\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.280817 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.280680 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9656\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-kube-api-access-v9656\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.280817 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.280702 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhvl8\" (UniqueName: \"kubernetes.io/projected/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-api-access-fhvl8\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:05:57.280817 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.280720 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e4de773-1f52-44a9-9077-1ae31610835b-cert\") pod \"ingress-canary-jjsrf\" (UID: \"9e4de773-1f52-44a9-9077-1ae31610835b\") " pod="openshift-ingress-canary/ingress-canary-jjsrf" Jul 13 17:05:57.280817 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.280759 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0807a42f-0465-4db7-8ce1-880139f0c35f-crio-socket\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:05:57.281354 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.280817 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c00d0e29-584d-4428-b879-f98edecd3b85-trusted-ca\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.281354 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.280883 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0807a42f-0465-4db7-8ce1-880139f0c35f-data-volume\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:05:57.281354 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.280905 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c00d0e29-584d-4428-b879-f98edecd3b85-ca-trust-extracted\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.281354 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.280922 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-config-volume\") pod \"dns-default-64h5v\" (UID: \"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a\") " pod="openshift-dns/dns-default-64h5v" Jul 13 17:05:57.281354 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.280941 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-tmp-dir\") pod \"dns-default-64h5v\" (UID: \"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a\") " pod="openshift-dns/dns-default-64h5v" Jul 13 17:05:57.281354 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.280964 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl7d6\" (UniqueName: \"kubernetes.io/projected/9e4de773-1f52-44a9-9077-1ae31610835b-kube-api-access-kl7d6\") pod \"ingress-canary-jjsrf\" (UID: \"9e4de773-1f52-44a9-9077-1ae31610835b\") " pod="openshift-ingress-canary/ingress-canary-jjsrf" Jul 13 17:05:57.281354 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.281034 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0807a42f-0465-4db7-8ce1-880139f0c35f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:05:57.281354 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.281063 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-registry-tls\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.281354 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.281088 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c00d0e29-584d-4428-b879-f98edecd3b85-installation-pull-secrets\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.281354 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.281112 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-metrics-tls\") pod \"dns-default-64h5v\" (UID: \"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a\") " pod="openshift-dns/dns-default-64h5v" Jul 13 17:05:57.281354 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.281153 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnjgc\" (UniqueName: \"kubernetes.io/projected/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-kube-api-access-lnjgc\") pod \"dns-default-64h5v\" (UID: \"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a\") " pod="openshift-dns/dns-default-64h5v" Jul 13 17:05:57.319768 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.319716 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:57.381946 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.381852 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0807a42f-0465-4db7-8ce1-880139f0c35f-data-volume\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:05:57.381946 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.381895 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c00d0e29-584d-4428-b879-f98edecd3b85-ca-trust-extracted\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.381946 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.381921 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-config-volume\") pod \"dns-default-64h5v\" (UID: \"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a\") " pod="openshift-dns/dns-default-64h5v" Jul 13 17:05:57.381946 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.381945 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-tmp-dir\") pod \"dns-default-64h5v\" (UID: \"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a\") " pod="openshift-dns/dns-default-64h5v" Jul 13 17:05:57.382252 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.381967 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl7d6\" (UniqueName: \"kubernetes.io/projected/9e4de773-1f52-44a9-9077-1ae31610835b-kube-api-access-kl7d6\") pod \"ingress-canary-jjsrf\" (UID: \"9e4de773-1f52-44a9-9077-1ae31610835b\") " pod="openshift-ingress-canary/ingress-canary-jjsrf" Jul 13 17:05:57.382252 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.382019 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0807a42f-0465-4db7-8ce1-880139f0c35f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:05:57.382252 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.382040 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-registry-tls\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.382252 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.382064 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c00d0e29-584d-4428-b879-f98edecd3b85-installation-pull-secrets\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.382252 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.382083 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-metrics-tls\") pod \"dns-default-64h5v\" (UID: \"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a\") " pod="openshift-dns/dns-default-64h5v" Jul 13 17:05:57.382252 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.382115 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnjgc\" (UniqueName: \"kubernetes.io/projected/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-kube-api-access-lnjgc\") pod \"dns-default-64h5v\" (UID: \"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a\") " pod="openshift-dns/dns-default-64h5v" Jul 13 17:05:57.382252 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.382149 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:05:57.382252 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.382180 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c00d0e29-584d-4428-b879-f98edecd3b85-image-registry-private-configuration\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.382252 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.382211 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c00d0e29-584d-4428-b879-f98edecd3b85-registry-certificates\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.382252 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.382238 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-bound-sa-token\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.382693 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.382269 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9656\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-kube-api-access-v9656\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.382693 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.382302 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhvl8\" (UniqueName: \"kubernetes.io/projected/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-api-access-fhvl8\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:05:57.382693 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.382312 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c00d0e29-584d-4428-b879-f98edecd3b85-ca-trust-extracted\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.382693 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.382331 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e4de773-1f52-44a9-9077-1ae31610835b-cert\") pod \"ingress-canary-jjsrf\" (UID: \"9e4de773-1f52-44a9-9077-1ae31610835b\") " pod="openshift-ingress-canary/ingress-canary-jjsrf" Jul 13 17:05:57.382693 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.382373 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0807a42f-0465-4db7-8ce1-880139f0c35f-crio-socket\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:05:57.382693 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.382268 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0807a42f-0465-4db7-8ce1-880139f0c35f-data-volume\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:05:57.382693 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.382429 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c00d0e29-584d-4428-b879-f98edecd3b85-trusted-ca\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.383103 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:57.382755 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:05:57.383103 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:57.382819 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-metrics-tls podName:3a8b92b2-17ca-4115-a0a1-6410f7fafb6a nodeName:}" failed. No retries permitted until 2026-07-13 17:05:57.882793584 +0000 UTC m=+35.545568769 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-metrics-tls") pod "dns-default-64h5v" (UID: "3a8b92b2-17ca-4115-a0a1-6410f7fafb6a") : secret "dns-default-metrics-tls" not found Jul 13 17:05:57.383103 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:57.382823 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 17:05:57.383103 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:57.382894 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-rbac-proxy-cm podName:0807a42f-0465-4db7-8ce1-880139f0c35f nodeName:}" failed. No retries permitted until 2026-07-13 17:05:57.882876393 +0000 UTC m=+35.545651575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qqmx6" (UID: "0807a42f-0465-4db7-8ce1-880139f0c35f") : configmap "kube-rbac-proxy" not found Jul 13 17:05:57.383832 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:57.383803 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 17:05:57.383964 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:57.383880 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0807a42f-0465-4db7-8ce1-880139f0c35f-insights-runtime-extractor-tls podName:0807a42f-0465-4db7-8ce1-880139f0c35f nodeName:}" failed. No retries permitted until 2026-07-13 17:05:57.883853119 +0000 UTC m=+35.546628297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0807a42f-0465-4db7-8ce1-880139f0c35f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qqmx6" (UID: "0807a42f-0465-4db7-8ce1-880139f0c35f") : secret "insights-runtime-extractor-tls" not found Jul 13 17:05:57.383964 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:57.383959 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 17:05:57.384080 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:57.383974 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d88ff76c8-2m949: secret "image-registry-tls" not found Jul 13 17:05:57.384080 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:57.384021 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-registry-tls podName:c00d0e29-584d-4428-b879-f98edecd3b85 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:57.884007988 +0000 UTC m=+35.546783154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-registry-tls") pod "image-registry-6d88ff76c8-2m949" (UID: "c00d0e29-584d-4428-b879-f98edecd3b85") : secret "image-registry-tls" not found Jul 13 17:05:57.384080 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.384067 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-tmp-dir\") pod \"dns-default-64h5v\" (UID: \"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a\") " pod="openshift-dns/dns-default-64h5v" Jul 13 17:05:57.384296 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.384273 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c00d0e29-584d-4428-b879-f98edecd3b85-registry-certificates\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.384363 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.384317 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-config-volume\") pod \"dns-default-64h5v\" (UID: \"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a\") " pod="openshift-dns/dns-default-64h5v" Jul 13 17:05:57.384607 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.384547 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0807a42f-0465-4db7-8ce1-880139f0c35f-crio-socket\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:05:57.385118 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.385095 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c00d0e29-584d-4428-b879-f98edecd3b85-trusted-ca\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.388637 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.387465 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c00d0e29-584d-4428-b879-f98edecd3b85-installation-pull-secrets\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.388637 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.387557 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c00d0e29-584d-4428-b879-f98edecd3b85-image-registry-private-configuration\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.389770 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.389721 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e4de773-1f52-44a9-9077-1ae31610835b-cert\") pod \"ingress-canary-jjsrf\" (UID: \"9e4de773-1f52-44a9-9077-1ae31610835b\") " pod="openshift-ingress-canary/ingress-canary-jjsrf" Jul 13 17:05:57.393673 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.393637 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9656\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-kube-api-access-v9656\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.394683 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.394631 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnjgc\" (UniqueName: \"kubernetes.io/projected/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-kube-api-access-lnjgc\") pod \"dns-default-64h5v\" (UID: \"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a\") " pod="openshift-dns/dns-default-64h5v" Jul 13 17:05:57.395235 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.395211 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-bound-sa-token\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.395401 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.395384 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhvl8\" (UniqueName: \"kubernetes.io/projected/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-api-access-fhvl8\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:05:57.397104 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.397073 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl7d6\" (UniqueName: \"kubernetes.io/projected/9e4de773-1f52-44a9-9077-1ae31610835b-kube-api-access-kl7d6\") pod \"ingress-canary-jjsrf\" (UID: \"9e4de773-1f52-44a9-9077-1ae31610835b\") " pod="openshift-ingress-canary/ingress-canary-jjsrf" Jul 13 17:05:57.420371 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.420329 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:57.509914 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.509878 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jjsrf" Jul 13 17:05:57.669450 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.669415 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:57.688705 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.688667 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jjsrf"] Jul 13 17:05:57.693672 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:05:57.693639 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e4de773_1f52_44a9_9077_1ae31610835b.slice/crio-6216d3b364b819509b3c991c72bf08e55c63ed337088bb6c5711fd92fc40f021 WatchSource:0}: Error finding container 6216d3b364b819509b3c991c72bf08e55c63ed337088bb6c5711fd92fc40f021: Status 404 returned error can't find the container with id 6216d3b364b819509b3c991c72bf08e55c63ed337088bb6c5711fd92fc40f021 Jul 13 17:05:57.770794 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.770761 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:57.844685 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.844651 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:57.890681 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.890566 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:05:57.890858 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:57.890707 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 17:05:57.890858 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.890720 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0807a42f-0465-4db7-8ce1-880139f0c35f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:05:57.890858 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:57.890801 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-rbac-proxy-cm podName:0807a42f-0465-4db7-8ce1-880139f0c35f nodeName:}" failed. No retries permitted until 2026-07-13 17:05:58.890776127 +0000 UTC m=+36.553551305 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qqmx6" (UID: "0807a42f-0465-4db7-8ce1-880139f0c35f") : configmap "kube-rbac-proxy" not found Jul 13 17:05:57.890858 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.890830 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-registry-tls\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:57.891050 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.890860 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-metrics-tls\") pod \"dns-default-64h5v\" (UID: \"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a\") " pod="openshift-dns/dns-default-64h5v" Jul 13 17:05:57.891050 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:57.890882 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 17:05:57.891050 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:57.890951 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0807a42f-0465-4db7-8ce1-880139f0c35f-insights-runtime-extractor-tls podName:0807a42f-0465-4db7-8ce1-880139f0c35f nodeName:}" failed. No retries permitted until 2026-07-13 17:05:58.890934328 +0000 UTC m=+36.553709514 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0807a42f-0465-4db7-8ce1-880139f0c35f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qqmx6" (UID: "0807a42f-0465-4db7-8ce1-880139f0c35f") : secret "insights-runtime-extractor-tls" not found Jul 13 17:05:57.891050 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:57.890991 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:05:57.891050 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:57.891017 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 17:05:57.891050 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:57.891028 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-metrics-tls podName:3a8b92b2-17ca-4115-a0a1-6410f7fafb6a nodeName:}" failed. No retries permitted until 2026-07-13 17:05:58.891016827 +0000 UTC m=+36.553791997 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-metrics-tls") pod "dns-default-64h5v" (UID: "3a8b92b2-17ca-4115-a0a1-6410f7fafb6a") : secret "dns-default-metrics-tls" not found Jul 13 17:05:57.891050 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:57.891029 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d88ff76c8-2m949: secret "image-registry-tls" not found Jul 13 17:05:57.891282 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:57.891062 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-registry-tls podName:c00d0e29-584d-4428-b879-f98edecd3b85 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:58.891054198 +0000 UTC m=+36.553829384 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-registry-tls") pod "image-registry-6d88ff76c8-2m949" (UID: "c00d0e29-584d-4428-b879-f98edecd3b85") : secret "image-registry-tls" not found Jul 13 17:05:57.955080 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.955047 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:05:57.955265 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.955090 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:05:57.955265 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.955141 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:05:57.958373 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.958349 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 17:05:57.958644 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.958627 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 17:05:57.959824 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.959804 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wvvhh\"" Jul 13 17:05:57.959923 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.959841 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5r2pv\"" Jul 13 17:05:57.960022 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.959987 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 17:05:57.960060 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:57.960022 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 17:05:58.100891 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:58.100849 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jjsrf" event={"ID":"9e4de773-1f52-44a9-9077-1ae31610835b","Type":"ContainerStarted","Data":"6216d3b364b819509b3c991c72bf08e55c63ed337088bb6c5711fd92fc40f021"} Jul 13 17:05:58.130444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:58.130363 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:58.227057 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:58.227022 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:58.586716 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:58.586624 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:58.728677 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:58.728635 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:58.871222 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:58.871132 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:58.900746 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:58.900688 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0807a42f-0465-4db7-8ce1-880139f0c35f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:05:58.900919 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:58.900753 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-registry-tls\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:05:58.900919 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:58.900783 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-metrics-tls\") pod \"dns-default-64h5v\" (UID: \"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a\") " pod="openshift-dns/dns-default-64h5v" Jul 13 17:05:58.900919 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:58.900831 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:05:58.900919 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:58.900866 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 17:05:58.900919 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:58.900911 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 17:05:58.901162 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:58.900925 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:05:58.901162 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:58.900931 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d88ff76c8-2m949: secret "image-registry-tls" not found Jul 13 17:05:58.901162 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:58.900950 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0807a42f-0465-4db7-8ce1-880139f0c35f-insights-runtime-extractor-tls podName:0807a42f-0465-4db7-8ce1-880139f0c35f nodeName:}" failed. No retries permitted until 2026-07-13 17:06:00.900932438 +0000 UTC m=+38.563707611 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0807a42f-0465-4db7-8ce1-880139f0c35f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qqmx6" (UID: "0807a42f-0465-4db7-8ce1-880139f0c35f") : secret "insights-runtime-extractor-tls" not found Jul 13 17:05:58.901162 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:58.900948 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 17:05:58.901162 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:58.900980 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-metrics-tls podName:3a8b92b2-17ca-4115-a0a1-6410f7fafb6a nodeName:}" failed. No retries permitted until 2026-07-13 17:06:00.900964095 +0000 UTC m=+38.563739287 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-metrics-tls") pod "dns-default-64h5v" (UID: "3a8b92b2-17ca-4115-a0a1-6410f7fafb6a") : secret "dns-default-metrics-tls" not found Jul 13 17:05:58.901162 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:58.901008 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-registry-tls podName:c00d0e29-584d-4428-b879-f98edecd3b85 nodeName:}" failed. No retries permitted until 2026-07-13 17:06:00.900998179 +0000 UTC m=+38.563773346 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-registry-tls") pod "image-registry-6d88ff76c8-2m949" (UID: "c00d0e29-584d-4428-b879-f98edecd3b85") : secret "image-registry-tls" not found Jul 13 17:05:58.901162 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:05:58.901028 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-rbac-proxy-cm podName:0807a42f-0465-4db7-8ce1-880139f0c35f nodeName:}" failed. No retries permitted until 2026-07-13 17:06:00.901020737 +0000 UTC m=+38.563795910 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qqmx6" (UID: "0807a42f-0465-4db7-8ce1-880139f0c35f") : configmap "kube-rbac-proxy" not found Jul 13 17:05:58.980392 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:58.980357 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:59.814357 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:05:59.814256 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:00.109893 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:00.109805 2564 generic.go:356] "Generic (PLEG): container finished" podID="e6411f69-741b-44f2-ae0f-7babdd39a621" containerID="00bd4142d5458b394780d57ae26a06e40b1c71ed55aab33a83d5f76826098791" exitCode=0 Jul 13 17:06:00.110050 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:00.109898 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc8ng" event={"ID":"e6411f69-741b-44f2-ae0f-7babdd39a621","Type":"ContainerDied","Data":"00bd4142d5458b394780d57ae26a06e40b1c71ed55aab33a83d5f76826098791"} Jul 13 17:06:00.209552 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:00.208986 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:00.477152 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:00.477097 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:00.601613 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:00.601581 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:00.915232 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:00.915201 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0807a42f-0465-4db7-8ce1-880139f0c35f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:06:00.915789 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:00.915238 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-registry-tls\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:06:00.915789 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:00.915269 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-metrics-tls\") pod \"dns-default-64h5v\" (UID: \"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a\") " pod="openshift-dns/dns-default-64h5v" Jul 13 17:06:00.915789 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:00.915327 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:06:00.915789 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:06:00.915398 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 17:06:00.915789 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:06:00.915427 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 17:06:00.915789 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:06:00.915400 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 17:06:00.915789 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:06:00.915483 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-rbac-proxy-cm podName:0807a42f-0465-4db7-8ce1-880139f0c35f nodeName:}" failed. No retries permitted until 2026-07-13 17:06:04.915465042 +0000 UTC m=+42.578240207 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qqmx6" (UID: "0807a42f-0465-4db7-8ce1-880139f0c35f") : configmap "kube-rbac-proxy" not found Jul 13 17:06:00.915789 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:06:00.915498 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0807a42f-0465-4db7-8ce1-880139f0c35f-insights-runtime-extractor-tls podName:0807a42f-0465-4db7-8ce1-880139f0c35f nodeName:}" failed. No retries permitted until 2026-07-13 17:06:04.915490969 +0000 UTC m=+42.578266146 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0807a42f-0465-4db7-8ce1-880139f0c35f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qqmx6" (UID: "0807a42f-0465-4db7-8ce1-880139f0c35f") : secret "insights-runtime-extractor-tls" not found Jul 13 17:06:00.915789 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:06:00.915484 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d88ff76c8-2m949: secret "image-registry-tls" not found Jul 13 17:06:00.915789 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:06:00.915403 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:06:00.915789 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:06:00.915531 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-registry-tls podName:c00d0e29-584d-4428-b879-f98edecd3b85 nodeName:}" failed. No retries permitted until 2026-07-13 17:06:04.915522988 +0000 UTC m=+42.578298154 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-registry-tls") pod "image-registry-6d88ff76c8-2m949" (UID: "c00d0e29-584d-4428-b879-f98edecd3b85") : secret "image-registry-tls" not found Jul 13 17:06:00.915789 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:06:00.915603 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-metrics-tls podName:3a8b92b2-17ca-4115-a0a1-6410f7fafb6a nodeName:}" failed. No retries permitted until 2026-07-13 17:06:04.915583102 +0000 UTC m=+42.578358270 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-metrics-tls") pod "dns-default-64h5v" (UID: "3a8b92b2-17ca-4115-a0a1-6410f7fafb6a") : secret "dns-default-metrics-tls" not found Jul 13 17:06:00.987508 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:00.987467 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:01.120153 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:01.120109 2564 generic.go:356] "Generic (PLEG): container finished" podID="e6411f69-741b-44f2-ae0f-7babdd39a621" containerID="89eb75b0728d4d7cbbca7f94a22a49b636cf590db0036cba373a87362a1aa1e0" exitCode=0 Jul 13 17:06:01.120306 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:01.120172 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc8ng" event={"ID":"e6411f69-741b-44f2-ae0f-7babdd39a621","Type":"ContainerDied","Data":"89eb75b0728d4d7cbbca7f94a22a49b636cf590db0036cba373a87362a1aa1e0"} Jul 13 17:06:01.584422 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:01.584372 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:01.984483 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:01.984454 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:02.126480 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:02.126432 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc8ng" event={"ID":"e6411f69-741b-44f2-ae0f-7babdd39a621","Type":"ContainerStarted","Data":"8970aac4a443ec64f1c316e212bf38296b639a9603fe7138dfdd3fbe5be88277"} Jul 13 17:06:02.127963 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:02.127878 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jjsrf" event={"ID":"9e4de773-1f52-44a9-9077-1ae31610835b","Type":"ContainerStarted","Data":"3484df00ecd1e83e4d49283c03cada4baa799ec13b30ab4a349a69eaaee7d41c"} Jul 13 17:06:02.158878 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:02.158824 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wc8ng" podStartSLOduration=7.612245784 podStartE2EDuration="39.15880835s" totalImagesPullingTime="31.546562566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:02.156759392 +0000 UTC m=+39.819534577" watchObservedRunningTime="2026-07-13 17:06:02.15880835 +0000 UTC m=+39.821583554" Jul 13 17:06:02.365672 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:02.365641 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:02.876012 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:02.875973 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:02.979334 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:02.979251 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:03.067538 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:03.067500 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:03.783023 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:03.782995 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:03.994014 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:03.993984 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:04.194963 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:04.194920 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:04.602150 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:04.602054 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:04.907711 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:04.907682 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:04.949827 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:04.949792 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:06:04.949941 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:04.949882 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0807a42f-0465-4db7-8ce1-880139f0c35f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:06:04.949941 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:04.949901 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-registry-tls\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:06:04.949941 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:06:04.949906 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 17:06:04.949941 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:04.949917 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-metrics-tls\") pod \"dns-default-64h5v\" (UID: \"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a\") " pod="openshift-dns/dns-default-64h5v" Jul 13 17:06:04.950155 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:06:04.949962 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-rbac-proxy-cm podName:0807a42f-0465-4db7-8ce1-880139f0c35f nodeName:}" failed. No retries permitted until 2026-07-13 17:06:12.949945601 +0000 UTC m=+50.612720767 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qqmx6" (UID: "0807a42f-0465-4db7-8ce1-880139f0c35f") : configmap "kube-rbac-proxy" not found Jul 13 17:06:04.950155 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:06:04.950119 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 17:06:04.950250 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:06:04.950193 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0807a42f-0465-4db7-8ce1-880139f0c35f-insights-runtime-extractor-tls podName:0807a42f-0465-4db7-8ce1-880139f0c35f nodeName:}" failed. No retries permitted until 2026-07-13 17:06:12.9501745 +0000 UTC m=+50.612949688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0807a42f-0465-4db7-8ce1-880139f0c35f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qqmx6" (UID: "0807a42f-0465-4db7-8ce1-880139f0c35f") : secret "insights-runtime-extractor-tls" not found Jul 13 17:06:04.953257 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:04.953233 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a8b92b2-17ca-4115-a0a1-6410f7fafb6a-metrics-tls\") pod \"dns-default-64h5v\" (UID: \"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a\") " pod="openshift-dns/dns-default-64h5v" Jul 13 17:06:04.953365 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:04.953301 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-registry-tls\") pod \"image-registry-6d88ff76c8-2m949\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:06:04.968582 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:04.968553 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:06:04.992471 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:04.992422 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-64h5v" Jul 13 17:06:05.128388 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:05.128336 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jjsrf" podStartSLOduration=3.945107348 podStartE2EDuration="8.128320436s" totalImagesPullingTime="4.183213088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:02.172630529 +0000 UTC m=+39.835405717" watchObservedRunningTime="2026-07-13 17:06:05.128320436 +0000 UTC m=+42.791095620" Jul 13 17:06:05.129102 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:05.129079 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d88ff76c8-2m949"] Jul 13 17:06:05.133121 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:06:05.133090 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc00d0e29_584d_4428_b879_f98edecd3b85.slice/crio-a1d4c3c35d9e66116d3626ae1e21526ea1b0a3fe23efcfb756f84082356b016a WatchSource:0}: Error finding container a1d4c3c35d9e66116d3626ae1e21526ea1b0a3fe23efcfb756f84082356b016a: Status 404 returned error can't find the container with id a1d4c3c35d9e66116d3626ae1e21526ea1b0a3fe23efcfb756f84082356b016a Jul 13 17:06:05.141989 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:05.141968 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-64h5v"] Jul 13 17:06:05.144869 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:06:05.144848 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a8b92b2_17ca_4115_a0a1_6410f7fafb6a.slice/crio-74f6263be2fd630094d3d6feb3d75cded94599a9bc9791488452b88a6e54ef24 WatchSource:0}: Error finding container 74f6263be2fd630094d3d6feb3d75cded94599a9bc9791488452b88a6e54ef24: Status 404 returned error can't find the container with id 74f6263be2fd630094d3d6feb3d75cded94599a9bc9791488452b88a6e54ef24 Jul 13 17:06:05.184450 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:05.184426 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:05.589355 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:05.589265 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:05.779296 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:05.779256 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:06.138488 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:06.138446 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-64h5v" event={"ID":"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a","Type":"ContainerStarted","Data":"74f6263be2fd630094d3d6feb3d75cded94599a9bc9791488452b88a6e54ef24"} Jul 13 17:06:06.139853 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:06.139829 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" event={"ID":"c00d0e29-584d-4428-b879-f98edecd3b85","Type":"ContainerStarted","Data":"87c6d2707ab88583e84b988da48c0078798c4d06840f401fcdbb2cd175da55f3"} Jul 13 17:06:06.139979 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:06.139863 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" event={"ID":"c00d0e29-584d-4428-b879-f98edecd3b85","Type":"ContainerStarted","Data":"a1d4c3c35d9e66116d3626ae1e21526ea1b0a3fe23efcfb756f84082356b016a"} Jul 13 17:06:06.139979 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:06.139954 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:06:06.161534 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:06.161484 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" podStartSLOduration=42.161469884 podStartE2EDuration="42.161469884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:06.161138295 +0000 UTC m=+43.823913483" watchObservedRunningTime="2026-07-13 17:06:06.161469884 +0000 UTC m=+43.824245072" Jul 13 17:06:06.385466 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:06.385433 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:06.985639 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:06.985606 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:07.191317 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:07.191281 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:07.384513 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:07.384308 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:07.785457 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:07.785419 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:08.149401 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:08.149369 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-64h5v" event={"ID":"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a","Type":"ContainerStarted","Data":"6a6ca794fcbccec5772a402f41036e7c5aad520f126244225ef7202b6ac53e6f"} Jul 13 17:06:08.149792 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:08.149421 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-64h5v" event={"ID":"3a8b92b2-17ca-4115-a0a1-6410f7fafb6a","Type":"ContainerStarted","Data":"23da16877a2b27dc4d8421643d027eec927ed43bf63bce9cd55cb84dee476224"} Jul 13 17:06:08.149792 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:08.149491 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-64h5v" Jul 13 17:06:08.165937 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:08.165889 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-64h5v" podStartSLOduration=8.577237523 podStartE2EDuration="11.16587431s" totalImagesPullingTime="2.588636787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:08.165523533 +0000 UTC m=+45.828298723" watchObservedRunningTime="2026-07-13 17:06:08.16587431 +0000 UTC m=+45.828649498" Jul 13 17:06:08.380227 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:08.380143 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:08.585017 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:08.584971 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:08.782054 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:08.782002 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:09.285745 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:09.285695 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:09.552688 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:09.552609 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:09.824220 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:09.824107 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:13.015455 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:13.015418 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0807a42f-0465-4db7-8ce1-880139f0c35f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:06:13.015944 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:13.015468 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:06:13.015998 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:13.015974 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0807a42f-0465-4db7-8ce1-880139f0c35f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:06:13.017914 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:13.017892 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0807a42f-0465-4db7-8ce1-880139f0c35f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qqmx6\" (UID: \"0807a42f-0465-4db7-8ce1-880139f0c35f\") " pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:06:13.099175 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:13.099138 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qqmx6" Jul 13 17:06:13.231081 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:13.231047 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qqmx6"] Jul 13 17:06:13.234243 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:06:13.234214 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0807a42f_0465_4db7_8ce1_880139f0c35f.slice/crio-3a1e012e83f74da47f27db780b5712ab2f568b5734deb6ab38c6461499c49d35 WatchSource:0}: Error finding container 3a1e012e83f74da47f27db780b5712ab2f568b5734deb6ab38c6461499c49d35: Status 404 returned error can't find the container with id 3a1e012e83f74da47f27db780b5712ab2f568b5734deb6ab38c6461499c49d35 Jul 13 17:06:14.167202 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:14.167162 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qqmx6" event={"ID":"0807a42f-0465-4db7-8ce1-880139f0c35f","Type":"ContainerStarted","Data":"3a1e012e83f74da47f27db780b5712ab2f568b5734deb6ab38c6461499c49d35"} Jul 13 17:06:17.652165 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:17.652123 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:17.768498 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:17.768464 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:17.934091 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:17.933999 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-pmzkl"] Jul 13 17:06:17.936269 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:17.936247 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-pmzkl" Jul 13 17:06:17.941878 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:17.941852 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 13 17:06:17.942015 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:17.941855 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-q9jpv\"" Jul 13 17:06:17.942437 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:17.942411 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 13 17:06:17.947601 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:17.947576 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d88ff76c8-2m949"] Jul 13 17:06:17.953771 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:17.953721 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps99k\" (UniqueName: \"kubernetes.io/projected/47c45b3d-1255-4dc7-ab98-97a443293a83-kube-api-access-ps99k\") pod \"downloads-68c9cbb497-pmzkl\" (UID: \"47c45b3d-1255-4dc7-ab98-97a443293a83\") " pod="openshift-console/downloads-68c9cbb497-pmzkl" Jul 13 17:06:17.959667 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:17.959641 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:17.964101 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:17.964064 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-pmzkl"] Jul 13 17:06:18.041797 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.041761 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-678db6845d-xhfqs"] Jul 13 17:06:18.044014 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.043988 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.054361 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.054321 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps99k\" (UniqueName: \"kubernetes.io/projected/47c45b3d-1255-4dc7-ab98-97a443293a83-kube-api-access-ps99k\") pod \"downloads-68c9cbb497-pmzkl\" (UID: \"47c45b3d-1255-4dc7-ab98-97a443293a83\") " pod="openshift-console/downloads-68c9cbb497-pmzkl" Jul 13 17:06:18.062766 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.062701 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-678db6845d-xhfqs"] Jul 13 17:06:18.079347 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.079319 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps99k\" (UniqueName: \"kubernetes.io/projected/47c45b3d-1255-4dc7-ab98-97a443293a83-kube-api-access-ps99k\") pod \"downloads-68c9cbb497-pmzkl\" (UID: \"47c45b3d-1255-4dc7-ab98-97a443293a83\") " pod="openshift-console/downloads-68c9cbb497-pmzkl" Jul 13 17:06:18.113647 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.113610 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:18.154796 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.154318 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-64h5v" Jul 13 17:06:18.154796 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.154783 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-image-registry-private-configuration\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.155022 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.154822 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-installation-pull-secrets\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.155022 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.154857 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-trusted-ca\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.155022 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.154900 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-registry-certificates\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.155022 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.154942 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-registry-tls\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.155022 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.155005 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-ca-trust-extracted\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.155251 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.155028 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxmnc\" (UniqueName: \"kubernetes.io/projected/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-kube-api-access-fxmnc\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.155251 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.155123 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-bound-sa-token\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.213808 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.212621 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:18.247698 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.247679 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-pmzkl" Jul 13 17:06:18.255539 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.255512 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-trusted-ca\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.255639 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.255565 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-registry-certificates\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.255639 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.255616 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-registry-tls\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.255847 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.255650 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-ca-trust-extracted\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.255847 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.255832 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxmnc\" (UniqueName: \"kubernetes.io/projected/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-kube-api-access-fxmnc\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.255981 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.255902 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-bound-sa-token\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.256037 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.256016 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-image-registry-private-configuration\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.256090 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.256041 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-installation-pull-secrets\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.256837 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.256294 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-ca-trust-extracted\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.256837 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.256778 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-trusted-ca\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.257509 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.257485 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-registry-certificates\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.259649 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.259621 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-image-registry-private-configuration\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.260041 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.260017 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-installation-pull-secrets\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.261485 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.261439 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-registry-tls\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.267437 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.267412 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-bound-sa-token\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.267709 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.267690 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxmnc\" (UniqueName: \"kubernetes.io/projected/3c5e9f64-15e5-484e-a78d-4f47e77f25d6-kube-api-access-fxmnc\") pod \"image-registry-678db6845d-xhfqs\" (UID: \"3c5e9f64-15e5-484e-a78d-4f47e77f25d6\") " pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.305216 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.305089 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-b7ptl"] Jul 13 17:06:18.305216 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.305154 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:18.309600 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.309575 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-b7ptl" Jul 13 17:06:18.312760 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.312519 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-fsn5d\"" Jul 13 17:06:18.312917 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.312895 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 13 17:06:18.320947 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.320924 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-b7ptl"] Jul 13 17:06:18.356246 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.356185 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:18.356435 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.356400 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d384e72f-14f4-4dae-a011-1c78c2700765-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-b7ptl\" (UID: \"d384e72f-14f4-4dae-a011-1c78c2700765\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-b7ptl" Jul 13 17:06:18.399121 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.399085 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-pmzkl"] Jul 13 17:06:18.401864 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:06:18.401814 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47c45b3d_1255_4dc7_ab98_97a443293a83.slice/crio-9ba77e8b583b829aabf6d4307f1d5e99bab1b6e67e6225540b378b27c97c2b7e WatchSource:0}: Error finding container 9ba77e8b583b829aabf6d4307f1d5e99bab1b6e67e6225540b378b27c97c2b7e: Status 404 returned error can't find the container with id 9ba77e8b583b829aabf6d4307f1d5e99bab1b6e67e6225540b378b27c97c2b7e Jul 13 17:06:18.402039 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.401888 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:18.457291 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.457261 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d384e72f-14f4-4dae-a011-1c78c2700765-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-b7ptl\" (UID: \"d384e72f-14f4-4dae-a011-1c78c2700765\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-b7ptl" Jul 13 17:06:18.457434 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:06:18.457413 2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 13 17:06:18.457497 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:06:18.457487 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d384e72f-14f4-4dae-a011-1c78c2700765-tls-certificates podName:d384e72f-14f4-4dae-a011-1c78c2700765 nodeName:}" failed. No retries permitted until 2026-07-13 17:06:18.957468699 +0000 UTC m=+56.620243883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/d384e72f-14f4-4dae-a011-1c78c2700765-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-b7ptl" (UID: "d384e72f-14f4-4dae-a011-1c78c2700765") : secret "prometheus-operator-admission-webhook-tls" not found Jul 13 17:06:18.492931 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.492858 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-678db6845d-xhfqs"] Jul 13 17:06:18.496082 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:06:18.496051 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c5e9f64_15e5_484e_a78d_4f47e77f25d6.slice/crio-12454e6a12300748c6d1f760f58ec0dece40a4a8483c20422185e0a6704968d7 WatchSource:0}: Error finding container 12454e6a12300748c6d1f760f58ec0dece40a4a8483c20422185e0a6704968d7: Status 404 returned error can't find the container with id 12454e6a12300748c6d1f760f58ec0dece40a4a8483c20422185e0a6704968d7 Jul 13 17:06:18.504835 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.504809 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:18.591387 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.591355 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:18.693190 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.693152 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:18.764132 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.764048 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:18.960908 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.960871 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d384e72f-14f4-4dae-a011-1c78c2700765-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-b7ptl\" (UID: \"d384e72f-14f4-4dae-a011-1c78c2700765\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-b7ptl" Jul 13 17:06:18.967052 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:18.967021 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d384e72f-14f4-4dae-a011-1c78c2700765-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-b7ptl\" (UID: \"d384e72f-14f4-4dae-a011-1c78c2700765\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-b7ptl" Jul 13 17:06:19.125938 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:19.125904 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:19.184054 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:19.183930 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-678db6845d-xhfqs" event={"ID":"3c5e9f64-15e5-484e-a78d-4f47e77f25d6","Type":"ContainerStarted","Data":"99645c7f586d920b730f0a0c52eadc471039c95e1789d1be5512364572e58519"} Jul 13 17:06:19.184054 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:19.183994 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-678db6845d-xhfqs" event={"ID":"3c5e9f64-15e5-484e-a78d-4f47e77f25d6","Type":"ContainerStarted","Data":"12454e6a12300748c6d1f760f58ec0dece40a4a8483c20422185e0a6704968d7"} Jul 13 17:06:19.184054 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:19.184023 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:19.185507 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:19.185475 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-pmzkl" event={"ID":"47c45b3d-1255-4dc7-ab98-97a443293a83","Type":"ContainerStarted","Data":"9ba77e8b583b829aabf6d4307f1d5e99bab1b6e67e6225540b378b27c97c2b7e"} Jul 13 17:06:19.187600 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:19.187565 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qqmx6" event={"ID":"0807a42f-0465-4db7-8ce1-880139f0c35f","Type":"ContainerStarted","Data":"aa667f7f8da31175aa6734a941892226c830422978164c5bb0a4c74fadb50138"} Jul 13 17:06:19.187698 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:19.187621 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qqmx6" event={"ID":"0807a42f-0465-4db7-8ce1-880139f0c35f","Type":"ContainerStarted","Data":"1090b7a85c08cdea42822350dbed5062740a82d3915b6a7747603d4526a69b3a"} Jul 13 17:06:19.195855 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:19.195787 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:19.209278 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:19.209219 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-678db6845d-xhfqs" podStartSLOduration=1.209199559 podStartE2EDuration="1.209199559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:06:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:19.208565248 +0000 UTC m=+56.871340435" watchObservedRunningTime="2026-07-13 17:06:19.209199559 +0000 UTC m=+56.871974748" Jul 13 17:06:19.223014 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:19.222981 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-b7ptl" Jul 13 17:06:19.322471 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:19.322439 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:19.363884 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:19.363854 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-b7ptl"] Jul 13 17:06:19.367075 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:06:19.367037 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd384e72f_14f4_4dae_a011_1c78c2700765.slice/crio-a2558a189115e5376cf6e0486f3938f5667e4109cd3d273201a61f34589c9b9b WatchSource:0}: Error finding container a2558a189115e5376cf6e0486f3938f5667e4109cd3d273201a61f34589c9b9b: Status 404 returned error can't find the container with id a2558a189115e5376cf6e0486f3938f5667e4109cd3d273201a61f34589c9b9b Jul 13 17:06:19.537149 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:19.537052 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:19.609054 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:19.608938 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:19.736534 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:19.736481 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:19.975207 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:19.975024 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:20.197867 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:20.197811 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-b7ptl" event={"ID":"d384e72f-14f4-4dae-a011-1c78c2700765","Type":"ContainerStarted","Data":"a2558a189115e5376cf6e0486f3938f5667e4109cd3d273201a61f34589c9b9b"} Jul 13 17:06:20.551688 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:20.551653 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:20.651864 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:20.651826 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:21.199645 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.199605 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-b7ptl" event={"ID":"d384e72f-14f4-4dae-a011-1c78c2700765","Type":"ContainerStarted","Data":"ae5e536e7a7a3d619994cece9b5f9b849712ab4fbd5a392c38f250580e4ec46c"} Jul 13 17:06:21.200127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.200062 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-b7ptl" Jul 13 17:06:21.206859 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.206831 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-b7ptl" Jul 13 17:06:21.216802 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.216739 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-b7ptl" podStartSLOduration=1.975958801 podStartE2EDuration="3.216706988s" totalImagesPullingTime="1.240748187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:21.215747271 +0000 UTC m=+58.878522455" watchObservedRunningTime="2026-07-13 17:06:21.216706988 +0000 UTC m=+58.879482177" Jul 13 17:06:21.327610 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.327556 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:21.366411 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.366368 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-zqrh4"] Jul 13 17:06:21.373803 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.373406 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-zqrh4" Jul 13 17:06:21.376955 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.376539 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 17:06:21.376955 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.376575 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 13 17:06:21.376955 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.376586 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 17:06:21.376955 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.376537 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 17:06:21.376955 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.376632 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 13 17:06:21.376955 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.376576 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-htb8j\"" Jul 13 17:06:21.380978 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.380957 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-zqrh4"] Jul 13 17:06:21.381975 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.381947 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x68lh\" (UniqueName: \"kubernetes.io/projected/d2341d47-022a-4f3f-bf9e-b8abb0caf1af-kube-api-access-x68lh\") pod \"prometheus-operator-64785b6675-zqrh4\" (UID: \"d2341d47-022a-4f3f-bf9e-b8abb0caf1af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zqrh4" Jul 13 17:06:21.382079 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.382018 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d2341d47-022a-4f3f-bf9e-b8abb0caf1af-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-zqrh4\" (UID: \"d2341d47-022a-4f3f-bf9e-b8abb0caf1af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zqrh4" Jul 13 17:06:21.382079 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.382059 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2341d47-022a-4f3f-bf9e-b8abb0caf1af-metrics-client-ca\") pod \"prometheus-operator-64785b6675-zqrh4\" (UID: \"d2341d47-022a-4f3f-bf9e-b8abb0caf1af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zqrh4" Jul 13 17:06:21.382203 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.382102 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2341d47-022a-4f3f-bf9e-b8abb0caf1af-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-zqrh4\" (UID: \"d2341d47-022a-4f3f-bf9e-b8abb0caf1af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zqrh4" Jul 13 17:06:21.400350 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.400298 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:21.460702 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.460607 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:21.482868 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.482827 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2341d47-022a-4f3f-bf9e-b8abb0caf1af-metrics-client-ca\") pod \"prometheus-operator-64785b6675-zqrh4\" (UID: \"d2341d47-022a-4f3f-bf9e-b8abb0caf1af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zqrh4" Jul 13 17:06:21.483064 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.482913 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2341d47-022a-4f3f-bf9e-b8abb0caf1af-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-zqrh4\" (UID: \"d2341d47-022a-4f3f-bf9e-b8abb0caf1af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zqrh4" Jul 13 17:06:21.483064 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.482971 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x68lh\" (UniqueName: \"kubernetes.io/projected/d2341d47-022a-4f3f-bf9e-b8abb0caf1af-kube-api-access-x68lh\") pod \"prometheus-operator-64785b6675-zqrh4\" (UID: \"d2341d47-022a-4f3f-bf9e-b8abb0caf1af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zqrh4" Jul 13 17:06:21.483064 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.483010 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d2341d47-022a-4f3f-bf9e-b8abb0caf1af-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-zqrh4\" (UID: \"d2341d47-022a-4f3f-bf9e-b8abb0caf1af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zqrh4" Jul 13 17:06:21.483638 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.483598 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2341d47-022a-4f3f-bf9e-b8abb0caf1af-metrics-client-ca\") pod \"prometheus-operator-64785b6675-zqrh4\" (UID: \"d2341d47-022a-4f3f-bf9e-b8abb0caf1af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zqrh4" Jul 13 17:06:21.485825 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.485795 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d2341d47-022a-4f3f-bf9e-b8abb0caf1af-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-zqrh4\" (UID: \"d2341d47-022a-4f3f-bf9e-b8abb0caf1af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zqrh4" Jul 13 17:06:21.485955 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.485831 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2341d47-022a-4f3f-bf9e-b8abb0caf1af-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-zqrh4\" (UID: \"d2341d47-022a-4f3f-bf9e-b8abb0caf1af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zqrh4" Jul 13 17:06:21.492198 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.492169 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x68lh\" (UniqueName: \"kubernetes.io/projected/d2341d47-022a-4f3f-bf9e-b8abb0caf1af-kube-api-access-x68lh\") pod \"prometheus-operator-64785b6675-zqrh4\" (UID: \"d2341d47-022a-4f3f-bf9e-b8abb0caf1af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zqrh4" Jul 13 17:06:21.686064 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:21.686018 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-zqrh4" Jul 13 17:06:22.096833 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:22.096715 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wmxtg" Jul 13 17:06:22.226129 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:22.226082 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-zqrh4"] Jul 13 17:06:22.228554 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:22.228529 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:22.229329 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:06:22.229299 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2341d47_022a_4f3f_bf9e_b8abb0caf1af.slice/crio-23d62ba36733d2091aa9b742c88a67d83f3d332f381d504e4d6f91b93c3f36bb WatchSource:0}: Error finding container 23d62ba36733d2091aa9b742c88a67d83f3d332f381d504e4d6f91b93c3f36bb: Status 404 returned error can't find the container with id 23d62ba36733d2091aa9b742c88a67d83f3d332f381d504e4d6f91b93c3f36bb Jul 13 17:06:22.288274 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:22.288228 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:23.209042 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:23.209000 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qqmx6" event={"ID":"0807a42f-0465-4db7-8ce1-880139f0c35f","Type":"ContainerStarted","Data":"9b3f1b8e0a92cfd9af66d73628e3db011f99fe1b7573c06242a8e03504634b5b"} Jul 13 17:06:23.210395 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:23.210363 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-zqrh4" event={"ID":"d2341d47-022a-4f3f-bf9e-b8abb0caf1af","Type":"ContainerStarted","Data":"23d62ba36733d2091aa9b742c88a67d83f3d332f381d504e4d6f91b93c3f36bb"} Jul 13 17:06:24.460121 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:24.460030 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:24.734839 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:24.734759 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:25.092489 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:25.092394 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:25.193019 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:25.192987 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:25.218775 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:25.218720 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-zqrh4" event={"ID":"d2341d47-022a-4f3f-bf9e-b8abb0caf1af","Type":"ContainerStarted","Data":"654e9831c61d435854a40b6202a1fd285c580a62621d473eaca0d99f89269987"} Jul 13 17:06:25.218913 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:25.218800 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-zqrh4" event={"ID":"d2341d47-022a-4f3f-bf9e-b8abb0caf1af","Type":"ContainerStarted","Data":"334ae3b08d5c85157308cfbd4706d7fd453c76775c696e395a26748b69eb4d74"} Jul 13 17:06:25.238431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:25.238357 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-zqrh4" podStartSLOduration=2.284854876 podStartE2EDuration="4.23833758s" totalImagesPullingTime="1.953482704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:06:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:25.237991829 +0000 UTC m=+62.900767019" watchObservedRunningTime="2026-07-13 17:06:25.23833758 +0000 UTC m=+62.901112753" Jul 13 17:06:25.239414 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:25.239375 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qqmx6" podStartSLOduration=19.475238418 podStartE2EDuration="28.239362107s" totalImagesPullingTime="8.764123689s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:23.229321913 +0000 UTC m=+60.892097101" watchObservedRunningTime="2026-07-13 17:06:25.239362107 +0000 UTC m=+62.902137295" Jul 13 17:06:25.293399 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:25.293355 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:26.309642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.309612 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:26.414710 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.414675 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:26.509477 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.509435 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:26.687130 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.687096 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:26.763320 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.763280 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mcn2q"] Jul 13 17:06:26.766161 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.766136 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.769090 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.768901 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 17:06:26.769090 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.768901 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tbg8b\"" Jul 13 17:06:26.769090 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.768945 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 17:06:26.769090 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.768969 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 17:06:26.773314 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.773292 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:26.829699 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.829662 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/83ae6253-ea82-42e8-9ae5-2fa29ada5204-node-exporter-wtmp\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.829901 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.829708 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/83ae6253-ea82-42e8-9ae5-2fa29ada5204-root\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.829901 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.829763 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/83ae6253-ea82-42e8-9ae5-2fa29ada5204-node-exporter-textfile\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.829901 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.829847 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83ae6253-ea82-42e8-9ae5-2fa29ada5204-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.829901 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.829876 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-547dd\" (UniqueName: \"kubernetes.io/projected/83ae6253-ea82-42e8-9ae5-2fa29ada5204-kube-api-access-547dd\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.830073 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.829945 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/83ae6253-ea82-42e8-9ae5-2fa29ada5204-sys\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.830073 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.829970 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83ae6253-ea82-42e8-9ae5-2fa29ada5204-metrics-client-ca\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.830073 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.830000 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/83ae6253-ea82-42e8-9ae5-2fa29ada5204-node-exporter-tls\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.830195 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.830074 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/83ae6253-ea82-42e8-9ae5-2fa29ada5204-node-exporter-accelerators-collector-config\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.834242 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.834214 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:26.930695 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.930659 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/83ae6253-ea82-42e8-9ae5-2fa29ada5204-node-exporter-accelerators-collector-config\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.930885 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.930710 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/83ae6253-ea82-42e8-9ae5-2fa29ada5204-node-exporter-wtmp\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.930885 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.930747 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/83ae6253-ea82-42e8-9ae5-2fa29ada5204-root\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.930885 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.930771 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/83ae6253-ea82-42e8-9ae5-2fa29ada5204-node-exporter-textfile\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.930885 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.930815 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83ae6253-ea82-42e8-9ae5-2fa29ada5204-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.930885 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.930839 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-547dd\" (UniqueName: \"kubernetes.io/projected/83ae6253-ea82-42e8-9ae5-2fa29ada5204-kube-api-access-547dd\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.930885 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.930874 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/83ae6253-ea82-42e8-9ae5-2fa29ada5204-sys\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.931173 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.930887 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/83ae6253-ea82-42e8-9ae5-2fa29ada5204-root\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.931173 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.930890 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/83ae6253-ea82-42e8-9ae5-2fa29ada5204-node-exporter-wtmp\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.931173 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.930919 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/83ae6253-ea82-42e8-9ae5-2fa29ada5204-sys\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.931173 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.931013 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83ae6253-ea82-42e8-9ae5-2fa29ada5204-metrics-client-ca\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.931173 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.931040 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/83ae6253-ea82-42e8-9ae5-2fa29ada5204-node-exporter-tls\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.931173 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:06:26.931147 2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 13 17:06:26.931387 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:06:26.931211 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83ae6253-ea82-42e8-9ae5-2fa29ada5204-node-exporter-tls podName:83ae6253-ea82-42e8-9ae5-2fa29ada5204 nodeName:}" failed. No retries permitted until 2026-07-13 17:06:27.431192452 +0000 UTC m=+65.093967618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/83ae6253-ea82-42e8-9ae5-2fa29ada5204-node-exporter-tls") pod "node-exporter-mcn2q" (UID: "83ae6253-ea82-42e8-9ae5-2fa29ada5204") : secret "node-exporter-tls" not found Jul 13 17:06:26.931387 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.931307 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/83ae6253-ea82-42e8-9ae5-2fa29ada5204-node-exporter-textfile\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.931387 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.931367 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/83ae6253-ea82-42e8-9ae5-2fa29ada5204-node-exporter-accelerators-collector-config\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.931592 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.931563 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83ae6253-ea82-42e8-9ae5-2fa29ada5204-metrics-client-ca\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.933835 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.933810 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83ae6253-ea82-42e8-9ae5-2fa29ada5204-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:26.940647 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:26.940583 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-547dd\" (UniqueName: \"kubernetes.io/projected/83ae6253-ea82-42e8-9ae5-2fa29ada5204-kube-api-access-547dd\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:27.085224 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.085185 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:27.187713 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.187679 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:27.288544 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.288463 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:27.360798 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.360761 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:27.436010 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.435971 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/83ae6253-ea82-42e8-9ae5-2fa29ada5204-node-exporter-tls\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:27.438675 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.438650 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/83ae6253-ea82-42e8-9ae5-2fa29ada5204-node-exporter-tls\") pod \"node-exporter-mcn2q\" (UID: \"83ae6253-ea82-42e8-9ae5-2fa29ada5204\") " pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:27.595140 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.595051 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:27.637744 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.637699 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs\") pod \"network-metrics-daemon-vpn5b\" (UID: \"738b8868-dacb-4206-bb1f-cbc6fdb57ecd\") " pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:06:27.641039 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.641004 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 17:06:27.651100 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.651073 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/738b8868-dacb-4206-bb1f-cbc6fdb57ecd-metrics-certs\") pod \"network-metrics-daemon-vpn5b\" (UID: \"738b8868-dacb-4206-bb1f-cbc6fdb57ecd\") " pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:06:27.676077 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.676034 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5r2pv\"" Jul 13 17:06:27.678167 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.678121 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mcn2q" Jul 13 17:06:27.684023 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.683996 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpn5b" Jul 13 17:06:27.690500 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:06:27.690467 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83ae6253_ea82_42e8_9ae5_2fa29ada5204.slice/crio-19c4e9fcc9f42745b5ddc3a62c7960413a30f688ebd0a9aa5e3f5c8b5dd891b7 WatchSource:0}: Error finding container 19c4e9fcc9f42745b5ddc3a62c7960413a30f688ebd0a9aa5e3f5c8b5dd891b7: Status 404 returned error can't find the container with id 19c4e9fcc9f42745b5ddc3a62c7960413a30f688ebd0a9aa5e3f5c8b5dd891b7 Jul 13 17:06:27.716050 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.715970 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:27.738302 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.738254 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret\") pod \"global-pull-secret-syncer-kj2dr\" (UID: \"8dcab0f9-5abb-4fa7-af28-54be545eb542\") " pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:06:27.741365 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.741340 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 17:06:27.752502 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.752435 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dcab0f9-5abb-4fa7-af28-54be545eb542-original-pull-secret\") pod \"global-pull-secret-syncer-kj2dr\" (UID: \"8dcab0f9-5abb-4fa7-af28-54be545eb542\") " pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:06:27.808044 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.807947 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:27.839164 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.839112 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwhr6\" (UniqueName: \"kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6\") pod \"network-check-target-jz68h\" (UID: \"cbe6b915-efe6-4616-b39e-cc1066ebd8e4\") " pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:06:27.842199 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.842171 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 17:06:27.846700 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.846648 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vpn5b"] Jul 13 17:06:27.849627 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:06:27.849388 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod738b8868_dacb_4206_bb1f_cbc6fdb57ecd.slice/crio-8cfc84de21f19b7706acb0bdfc33636cc666bcbaa971c514713c5ce46663db27 WatchSource:0}: Error finding container 8cfc84de21f19b7706acb0bdfc33636cc666bcbaa971c514713c5ce46663db27: Status 404 returned error can't find the container with id 8cfc84de21f19b7706acb0bdfc33636cc666bcbaa971c514713c5ce46663db27 Jul 13 17:06:27.852465 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.852446 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 17:06:27.862791 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.862770 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwhr6\" (UniqueName: \"kubernetes.io/projected/cbe6b915-efe6-4616-b39e-cc1066ebd8e4-kube-api-access-mwhr6\") pod \"network-check-target-jz68h\" (UID: \"cbe6b915-efe6-4616-b39e-cc1066ebd8e4\") " pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:06:27.953299 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.953272 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:06:27.969059 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.969027 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wvvhh\"" Jul 13 17:06:27.976657 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.976634 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:06:27.979011 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:27.978987 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj2dr" Jul 13 17:06:28.002055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:28.002012 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:28.164118 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:28.163976 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jz68h"] Jul 13 17:06:28.167963 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:06:28.167925 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbe6b915_efe6_4616_b39e_cc1066ebd8e4.slice/crio-2aca286a9cf3f0abfdbe6f486326f14171efdf3055bdbdf5f8fcf32fd469b825 WatchSource:0}: Error finding container 2aca286a9cf3f0abfdbe6f486326f14171efdf3055bdbdf5f8fcf32fd469b825: Status 404 returned error can't find the container with id 2aca286a9cf3f0abfdbe6f486326f14171efdf3055bdbdf5f8fcf32fd469b825 Jul 13 17:06:28.190219 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:28.190184 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kj2dr"] Jul 13 17:06:28.196606 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:06:28.196576 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dcab0f9_5abb_4fa7_af28_54be545eb542.slice/crio-f3ac039c1d241655e5ba1e51c84d187f35b70688f0d4126b51f87a4c01e13815 WatchSource:0}: Error finding container f3ac039c1d241655e5ba1e51c84d187f35b70688f0d4126b51f87a4c01e13815: Status 404 returned error can't find the container with id f3ac039c1d241655e5ba1e51c84d187f35b70688f0d4126b51f87a4c01e13815 Jul 13 17:06:28.230930 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:28.230887 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kj2dr" event={"ID":"8dcab0f9-5abb-4fa7-af28-54be545eb542","Type":"ContainerStarted","Data":"f3ac039c1d241655e5ba1e51c84d187f35b70688f0d4126b51f87a4c01e13815"} Jul 13 17:06:28.231988 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:28.231960 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jz68h" event={"ID":"cbe6b915-efe6-4616-b39e-cc1066ebd8e4","Type":"ContainerStarted","Data":"2aca286a9cf3f0abfdbe6f486326f14171efdf3055bdbdf5f8fcf32fd469b825"} Jul 13 17:06:28.233182 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:28.233160 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vpn5b" event={"ID":"738b8868-dacb-4206-bb1f-cbc6fdb57ecd","Type":"ContainerStarted","Data":"8cfc84de21f19b7706acb0bdfc33636cc666bcbaa971c514713c5ce46663db27"} Jul 13 17:06:28.240022 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:28.239994 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mcn2q" event={"ID":"83ae6253-ea82-42e8-9ae5-2fa29ada5204","Type":"ContainerStarted","Data":"19c4e9fcc9f42745b5ddc3a62c7960413a30f688ebd0a9aa5e3f5c8b5dd891b7"} Jul 13 17:06:29.248211 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:29.248079 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vpn5b" event={"ID":"738b8868-dacb-4206-bb1f-cbc6fdb57ecd","Type":"ContainerStarted","Data":"e24064d03920919ff5fce8cb6bc10b678fdb719c7d1b8596065bbf2a99986b5c"} Jul 13 17:06:29.248211 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:29.248152 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vpn5b" event={"ID":"738b8868-dacb-4206-bb1f-cbc6fdb57ecd","Type":"ContainerStarted","Data":"62bcdb76775a175ea739514b49b2f15a1c6b9d6aec15b322ca05b8d92308fa3f"} Jul 13 17:06:29.271257 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:29.270363 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vpn5b" podStartSLOduration=66.291327083 podStartE2EDuration="1m7.270340413s" totalImagesPullingTime="979.01333ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:29.268450652 +0000 UTC m=+66.931225852" watchObservedRunningTime="2026-07-13 17:06:29.270340413 +0000 UTC m=+66.933115603" Jul 13 17:06:29.392876 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:29.392801 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:29.490887 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:29.490801 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:29.764020 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:29.763932 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:29.829813 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:29.829777 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:29.908547 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:29.908428 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:30.110632 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:30.110603 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:30.262784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:30.262694 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mcn2q" event={"ID":"83ae6253-ea82-42e8-9ae5-2fa29ada5204","Type":"ContainerStarted","Data":"8fb24e2a08a6d4565734f4a75bf86b36b4fc383714f28c5a8ee7aca4b5552763"} Jul 13 17:06:31.268998 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:31.268961 2564 generic.go:356] "Generic (PLEG): container finished" podID="83ae6253-ea82-42e8-9ae5-2fa29ada5204" containerID="8fb24e2a08a6d4565734f4a75bf86b36b4fc383714f28c5a8ee7aca4b5552763" exitCode=0 Jul 13 17:06:31.269443 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:31.269069 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mcn2q" event={"ID":"83ae6253-ea82-42e8-9ae5-2fa29ada5204","Type":"ContainerDied","Data":"8fb24e2a08a6d4565734f4a75bf86b36b4fc383714f28c5a8ee7aca4b5552763"} Jul 13 17:06:31.269443 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:31.269132 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mcn2q" event={"ID":"83ae6253-ea82-42e8-9ae5-2fa29ada5204","Type":"ContainerStarted","Data":"bd42f365fed7e95baed4d4a8df648b7363c64d184ad5ceb25b6515c4f64e7e42"} Jul 13 17:06:31.269443 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:31.269165 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mcn2q" event={"ID":"83ae6253-ea82-42e8-9ae5-2fa29ada5204","Type":"ContainerStarted","Data":"a4818bbad0b033457adbfde21206ad6827862d6e5d44f4d7dc453b4f6a6c9edd"} Jul 13 17:06:31.296110 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:31.296053 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mcn2q" podStartSLOduration=2.890801969 podStartE2EDuration="5.296030737s" totalImagesPullingTime="2.284604143s" totalInitContainerRuntime="120.624625ms" isStatefulPod=true podCreationTimestamp="2026-07-13 17:06:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:31.295367822 +0000 UTC m=+68.958143009" watchObservedRunningTime="2026-07-13 17:06:31.296030737 +0000 UTC m=+68.958805927" Jul 13 17:06:31.505296 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:31.505259 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:31.569867 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:31.569778 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:31.831339 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:31.831259 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:31.888693 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:31.888650 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:32.191600 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:32.191564 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:32.332787 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:32.332745 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:32.431381 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:32.431343 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:32.569876 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:32.569782 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:32.634587 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:32.634547 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:33.023254 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:33.020338 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:36.159218 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.159183 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b6997cb96-zqv5g"] Jul 13 17:06:36.162488 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.162462 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.165508 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.165485 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 17:06:36.165659 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.165643 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 17:06:36.165722 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.165674 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 17:06:36.167193 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.167086 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 17:06:36.167328 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.167299 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 17:06:36.167747 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.167717 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-bfd9n\"" Jul 13 17:06:36.173249 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.173213 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 17:06:36.174040 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.174020 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b6997cb96-zqv5g"] Jul 13 17:06:36.210357 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.210327 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-oauth-serving-cert\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.210538 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.210381 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/245d8353-8f05-4fe3-86d0-46c66008a8fc-console-serving-cert\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.210538 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.210419 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/245d8353-8f05-4fe3-86d0-46c66008a8fc-console-oauth-config\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.210538 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.210440 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-trusted-ca-bundle\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.210538 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.210504 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87h8v\" (UniqueName: \"kubernetes.io/projected/245d8353-8f05-4fe3-86d0-46c66008a8fc-kube-api-access-87h8v\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.210779 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.210550 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-console-config\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.210779 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.210619 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-service-ca\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.311219 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.311183 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-service-ca\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.311423 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.311243 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-oauth-serving-cert\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.311423 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.311281 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/245d8353-8f05-4fe3-86d0-46c66008a8fc-console-serving-cert\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.311423 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.311314 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/245d8353-8f05-4fe3-86d0-46c66008a8fc-console-oauth-config\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.311423 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.311337 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-trusted-ca-bundle\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.311423 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.311363 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87h8v\" (UniqueName: \"kubernetes.io/projected/245d8353-8f05-4fe3-86d0-46c66008a8fc-kube-api-access-87h8v\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.311423 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.311396 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-console-config\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.318172 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.312466 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-service-ca\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.318172 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.312904 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-oauth-serving-cert\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.318172 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.313505 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-console-config\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.318172 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.313532 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-trusted-ca-bundle\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.318172 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.314981 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/245d8353-8f05-4fe3-86d0-46c66008a8fc-console-oauth-config\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.318172 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.315655 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/245d8353-8f05-4fe3-86d0-46c66008a8fc-console-serving-cert\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.321321 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.321296 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87h8v\" (UniqueName: \"kubernetes.io/projected/245d8353-8f05-4fe3-86d0-46c66008a8fc-kube-api-access-87h8v\") pod \"console-7b6997cb96-zqv5g\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:36.349980 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.349945 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:36.450804 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.450690 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:36.473749 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:36.473687 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:37.844950 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:37.844833 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:37.944928 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:37.944883 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:40.207118 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:40.207089 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-678db6845d-xhfqs" Jul 13 17:06:40.360303 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:40.360250 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:40.452873 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:40.452828 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:40.557118 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:40.557026 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:41.063512 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:41.063466 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:41.171184 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:41.171117 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:41.255174 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:41.255097 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b6997cb96-zqv5g"] Jul 13 17:06:41.258025 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:41.257996 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:41.258952 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:06:41.258931 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod245d8353_8f05_4fe3_86d0_46c66008a8fc.slice/crio-99591dd7597f14172cedd88b8a8a425948cddad077b59d6e5350ccab9d4a0b21 WatchSource:0}: Error finding container 99591dd7597f14172cedd88b8a8a425948cddad077b59d6e5350ccab9d4a0b21: Status 404 returned error can't find the container with id 99591dd7597f14172cedd88b8a8a425948cddad077b59d6e5350ccab9d4a0b21 Jul 13 17:06:41.310664 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:41.310569 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kj2dr" event={"ID":"8dcab0f9-5abb-4fa7-af28-54be545eb542","Type":"ContainerStarted","Data":"8ab8a7ab8a4343394cf2dabf8669da0bb6f6c66d1839f40e0b480838029418bf"} Jul 13 17:06:41.312572 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:41.312541 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jz68h" event={"ID":"cbe6b915-efe6-4616-b39e-cc1066ebd8e4","Type":"ContainerStarted","Data":"f2bfb645233e8728f3af05ade0336f6b491d2da46bdee1b9be54f151cb7ee2c5"} Jul 13 17:06:41.312713 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:41.312598 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:06:41.313943 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:41.313914 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b6997cb96-zqv5g" event={"ID":"245d8353-8f05-4fe3-86d0-46c66008a8fc","Type":"ContainerStarted","Data":"99591dd7597f14172cedd88b8a8a425948cddad077b59d6e5350ccab9d4a0b21"} Jul 13 17:06:41.315554 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:41.315533 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-pmzkl" event={"ID":"47c45b3d-1255-4dc7-ab98-97a443293a83","Type":"ContainerStarted","Data":"8bcb9bcbd551a9e46c7cec698b56b54c5af564d7485ef27bf0052157b197d755"} Jul 13 17:06:41.315812 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:41.315767 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-pmzkl" Jul 13 17:06:41.317435 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:41.317411 2564 patch_prober.go:28] interesting pod/downloads-68c9cbb497-pmzkl container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.10:8080/\": dial tcp 10.133.0.10:8080: connect: connection refused" start-of-body= Jul 13 17:06:41.317535 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:41.317461 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-68c9cbb497-pmzkl" podUID="47c45b3d-1255-4dc7-ab98-97a443293a83" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.10:8080/\": dial tcp 10.133.0.10:8080: connect: connection refused" Jul 13 17:06:41.346505 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:41.346454 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kj2dr" podStartSLOduration=65.433314168 podStartE2EDuration="1m18.346435158s" totalImagesPullingTime="12.91312099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:41.326902614 +0000 UTC m=+78.989677806" watchObservedRunningTime="2026-07-13 17:06:41.346435158 +0000 UTC m=+79.009210337" Jul 13 17:06:41.347270 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:41.347226 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-pmzkl" podStartSLOduration=1.651334184 podStartE2EDuration="24.347215617s" totalImagesPullingTime="22.695881433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:41.346944897 +0000 UTC m=+79.009720086" watchObservedRunningTime="2026-07-13 17:06:41.347215617 +0000 UTC m=+79.009990805" Jul 13 17:06:41.362502 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:41.362442 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jz68h" podStartSLOduration=65.416133787 podStartE2EDuration="1m18.362422223s" totalImagesPullingTime="12.946288436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:41.361604561 +0000 UTC m=+79.024379749" watchObservedRunningTime="2026-07-13 17:06:41.362422223 +0000 UTC m=+79.025197412" Jul 13 17:06:42.333438 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:42.333403 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-pmzkl" Jul 13 17:06:42.972414 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:42.972325 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" podUID="c00d0e29-584d-4428-b879-f98edecd3b85" containerName="registry" containerID="cri-o://87c6d2707ab88583e84b988da48c0078798c4d06840f401fcdbb2cd175da55f3" gracePeriod=30 Jul 13 17:06:43.326664 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:43.326424 2564 generic.go:356] "Generic (PLEG): container finished" podID="c00d0e29-584d-4428-b879-f98edecd3b85" containerID="87c6d2707ab88583e84b988da48c0078798c4d06840f401fcdbb2cd175da55f3" exitCode=0 Jul 13 17:06:43.326840 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:43.326775 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" event={"ID":"c00d0e29-584d-4428-b879-f98edecd3b85","Type":"ContainerDied","Data":"87c6d2707ab88583e84b988da48c0078798c4d06840f401fcdbb2cd175da55f3"} Jul 13 17:06:44.566273 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.565967 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:06:44.686373 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.686330 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c00d0e29-584d-4428-b879-f98edecd3b85-installation-pull-secrets\") pod \"c00d0e29-584d-4428-b879-f98edecd3b85\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " Jul 13 17:06:44.686561 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.686390 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c00d0e29-584d-4428-b879-f98edecd3b85-ca-trust-extracted\") pod \"c00d0e29-584d-4428-b879-f98edecd3b85\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " Jul 13 17:06:44.686561 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.686422 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-bound-sa-token\") pod \"c00d0e29-584d-4428-b879-f98edecd3b85\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " Jul 13 17:06:44.686561 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.686456 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9656\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-kube-api-access-v9656\") pod \"c00d0e29-584d-4428-b879-f98edecd3b85\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " Jul 13 17:06:44.686561 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.686524 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c00d0e29-584d-4428-b879-f98edecd3b85-trusted-ca\") pod \"c00d0e29-584d-4428-b879-f98edecd3b85\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " Jul 13 17:06:44.686796 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.686662 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c00d0e29-584d-4428-b879-f98edecd3b85-image-registry-private-configuration\") pod \"c00d0e29-584d-4428-b879-f98edecd3b85\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " Jul 13 17:06:44.686796 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.686724 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c00d0e29-584d-4428-b879-f98edecd3b85-registry-certificates\") pod \"c00d0e29-584d-4428-b879-f98edecd3b85\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " Jul 13 17:06:44.686796 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.686782 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-registry-tls\") pod \"c00d0e29-584d-4428-b879-f98edecd3b85\" (UID: \"c00d0e29-584d-4428-b879-f98edecd3b85\") " Jul 13 17:06:44.686951 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.686887 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c00d0e29-584d-4428-b879-f98edecd3b85-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c00d0e29-584d-4428-b879-f98edecd3b85" (UID: "c00d0e29-584d-4428-b879-f98edecd3b85"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:06:44.687047 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.687027 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c00d0e29-584d-4428-b879-f98edecd3b85-trusted-ca\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:06:44.687184 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.687075 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c00d0e29-584d-4428-b879-f98edecd3b85-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c00d0e29-584d-4428-b879-f98edecd3b85" (UID: "c00d0e29-584d-4428-b879-f98edecd3b85"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:06:44.689422 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.689392 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-kube-api-access-v9656" (OuterVolumeSpecName: "kube-api-access-v9656") pod "c00d0e29-584d-4428-b879-f98edecd3b85" (UID: "c00d0e29-584d-4428-b879-f98edecd3b85"). InnerVolumeSpecName "kube-api-access-v9656". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:06:44.689604 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.689557 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c00d0e29-584d-4428-b879-f98edecd3b85" (UID: "c00d0e29-584d-4428-b879-f98edecd3b85"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:06:44.689709 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.689643 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c00d0e29-584d-4428-b879-f98edecd3b85-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c00d0e29-584d-4428-b879-f98edecd3b85" (UID: "c00d0e29-584d-4428-b879-f98edecd3b85"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:06:44.689858 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.689826 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c00d0e29-584d-4428-b879-f98edecd3b85-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c00d0e29-584d-4428-b879-f98edecd3b85" (UID: "c00d0e29-584d-4428-b879-f98edecd3b85"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:06:44.690136 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.690111 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c00d0e29-584d-4428-b879-f98edecd3b85" (UID: "c00d0e29-584d-4428-b879-f98edecd3b85"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:06:44.696377 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.696353 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c00d0e29-584d-4428-b879-f98edecd3b85-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c00d0e29-584d-4428-b879-f98edecd3b85" (UID: "c00d0e29-584d-4428-b879-f98edecd3b85"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:06:44.788192 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.787744 2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c00d0e29-584d-4428-b879-f98edecd3b85-installation-pull-secrets\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:06:44.788412 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.788393 2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c00d0e29-584d-4428-b879-f98edecd3b85-ca-trust-extracted\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:06:44.788513 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.788500 2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-bound-sa-token\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:06:44.788593 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.788582 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9656\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-kube-api-access-v9656\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:06:44.788690 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.788677 2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c00d0e29-584d-4428-b879-f98edecd3b85-image-registry-private-configuration\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:06:44.788788 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.788775 2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c00d0e29-584d-4428-b879-f98edecd3b85-registry-certificates\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:06:44.788878 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:44.788867 2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c00d0e29-584d-4428-b879-f98edecd3b85-registry-tls\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:06:45.338014 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:45.337920 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b6997cb96-zqv5g" event={"ID":"245d8353-8f05-4fe3-86d0-46c66008a8fc","Type":"ContainerStarted","Data":"2f9f4b6aefddbf671fda9dd907f2f9e14b43e093d16bdcae8cfcec09633e8bfc"} Jul 13 17:06:45.339539 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:45.339507 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" event={"ID":"c00d0e29-584d-4428-b879-f98edecd3b85","Type":"ContainerDied","Data":"a1d4c3c35d9e66116d3626ae1e21526ea1b0a3fe23efcfb756f84082356b016a"} Jul 13 17:06:45.339667 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:45.339553 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d88ff76c8-2m949" Jul 13 17:06:45.339667 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:45.339568 2564 scope.go:117] "RemoveContainer" containerID="87c6d2707ab88583e84b988da48c0078798c4d06840f401fcdbb2cd175da55f3" Jul 13 17:06:45.391721 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:45.391670 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b6997cb96-zqv5g" podStartSLOduration=5.687130554 podStartE2EDuration="9.391653296s" totalImagesPullingTime="3.704522742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:45.36920884 +0000 UTC m=+83.031984031" watchObservedRunningTime="2026-07-13 17:06:45.391653296 +0000 UTC m=+83.054428484" Jul 13 17:06:45.392099 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:45.392077 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d88ff76c8-2m949"] Jul 13 17:06:45.396882 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:45.396843 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:45.398104 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:45.398077 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6d88ff76c8-2m949"] Jul 13 17:06:45.489719 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:45.489683 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:45.777327 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:45.777285 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:45.878451 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:45.878400 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:46.474901 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:46.474864 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:46.475091 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:46.474928 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:46.480706 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:46.480678 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:46.961177 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:46.961142 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c00d0e29-584d-4428-b879-f98edecd3b85" path="/var/lib/kubelet/pods/c00d0e29-584d-4428-b879-f98edecd3b85/volumes" Jul 13 17:06:47.356457 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:47.356337 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:06:47.594550 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:47.594506 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:47.693266 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:47.693214 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:48.138184 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:48.138072 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:48.237985 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:48.237945 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:50.167177 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:50.167129 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:50.265582 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:50.265529 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:50.365327 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:50.365288 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:51.193116 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:51.193083 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:06:51.291280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:06:51.291245 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:07:03.070787 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:03.070752 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:07:03.172641 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:03.172605 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:07:06.878448 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:06.878404 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:07:06.987253 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:06.987217 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:07:08.833264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:08.833226 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:07:12.324245 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:12.324213 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jz68h" Jul 13 17:07:18.209034 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:18.208979 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:07:37.962857 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:37.962803 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:07:46.075530 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:46.075468 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:07:46.277697 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:46.277663 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:07:51.162643 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.162607 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-76dc565d9f-w97hl"] Jul 13 17:07:51.163507 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.163483 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c00d0e29-584d-4428-b879-f98edecd3b85" containerName="registry" Jul 13 17:07:51.163630 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.163619 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c00d0e29-584d-4428-b879-f98edecd3b85" containerName="registry" Jul 13 17:07:51.163813 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.163800 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c00d0e29-584d-4428-b879-f98edecd3b85" containerName="registry" Jul 13 17:07:51.169723 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.169693 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.173170 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.173138 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 13 17:07:51.173533 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.173498 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-kz8vs\"" Jul 13 17:07:51.173647 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.173588 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 13 17:07:51.173647 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.173612 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 13 17:07:51.173647 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.173638 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 13 17:07:51.173647 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.173595 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 13 17:07:51.181153 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.181126 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-76dc565d9f-w97hl"] Jul 13 17:07:51.182025 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.181834 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 13 17:07:51.271032 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.270998 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57625ec7-ea64-4acd-a884-68fec13cadbe-serving-certs-ca-bundle\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.271239 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.271052 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/57625ec7-ea64-4acd-a884-68fec13cadbe-secret-telemeter-client\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.271239 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.271079 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/57625ec7-ea64-4acd-a884-68fec13cadbe-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.271239 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.271105 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57625ec7-ea64-4acd-a884-68fec13cadbe-metrics-client-ca\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.271239 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.271136 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/57625ec7-ea64-4acd-a884-68fec13cadbe-telemeter-client-tls\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.271239 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.271157 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/57625ec7-ea64-4acd-a884-68fec13cadbe-federate-client-tls\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.271239 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.271178 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjfbd\" (UniqueName: \"kubernetes.io/projected/57625ec7-ea64-4acd-a884-68fec13cadbe-kube-api-access-rjfbd\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.271239 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.271205 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57625ec7-ea64-4acd-a884-68fec13cadbe-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.372151 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.372112 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/57625ec7-ea64-4acd-a884-68fec13cadbe-secret-telemeter-client\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.372151 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.372156 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/57625ec7-ea64-4acd-a884-68fec13cadbe-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.372475 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.372175 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57625ec7-ea64-4acd-a884-68fec13cadbe-metrics-client-ca\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.372475 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.372203 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/57625ec7-ea64-4acd-a884-68fec13cadbe-telemeter-client-tls\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.372475 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.372229 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/57625ec7-ea64-4acd-a884-68fec13cadbe-federate-client-tls\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.372475 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.372253 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjfbd\" (UniqueName: \"kubernetes.io/projected/57625ec7-ea64-4acd-a884-68fec13cadbe-kube-api-access-rjfbd\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.372475 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.372276 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57625ec7-ea64-4acd-a884-68fec13cadbe-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.372475 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.372329 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57625ec7-ea64-4acd-a884-68fec13cadbe-serving-certs-ca-bundle\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.373040 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.373013 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57625ec7-ea64-4acd-a884-68fec13cadbe-serving-certs-ca-bundle\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.373165 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.373013 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57625ec7-ea64-4acd-a884-68fec13cadbe-metrics-client-ca\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.373558 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.373534 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57625ec7-ea64-4acd-a884-68fec13cadbe-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.375504 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.375479 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/57625ec7-ea64-4acd-a884-68fec13cadbe-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.375752 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.375708 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/57625ec7-ea64-4acd-a884-68fec13cadbe-federate-client-tls\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.375752 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.375741 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/57625ec7-ea64-4acd-a884-68fec13cadbe-telemeter-client-tls\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.375869 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.375751 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/57625ec7-ea64-4acd-a884-68fec13cadbe-secret-telemeter-client\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.381766 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.381711 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjfbd\" (UniqueName: \"kubernetes.io/projected/57625ec7-ea64-4acd-a884-68fec13cadbe-kube-api-access-rjfbd\") pod \"telemeter-client-76dc565d9f-w97hl\" (UID: \"57625ec7-ea64-4acd-a884-68fec13cadbe\") " pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.484147 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.484047 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" Jul 13 17:07:51.640519 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:51.640244 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-76dc565d9f-w97hl"] Jul 13 17:07:51.643252 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:07:51.643217 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57625ec7_ea64_4acd_a884_68fec13cadbe.slice/crio-32a4422e021055e3435c07a53f266e77f978367ad436f346767291f1347b6fa8 WatchSource:0}: Error finding container 32a4422e021055e3435c07a53f266e77f978367ad436f346767291f1347b6fa8: Status 404 returned error can't find the container with id 32a4422e021055e3435c07a53f266e77f978367ad436f346767291f1347b6fa8 Jul 13 17:07:52.584149 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:52.584111 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" event={"ID":"57625ec7-ea64-4acd-a884-68fec13cadbe","Type":"ContainerStarted","Data":"32a4422e021055e3435c07a53f266e77f978367ad436f346767291f1347b6fa8"} Jul 13 17:07:54.593806 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:54.593770 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" event={"ID":"57625ec7-ea64-4acd-a884-68fec13cadbe","Type":"ContainerStarted","Data":"c1625819e25df3092d59f0a8ec6c17dd2d9dafc88aca6880f73a833392a2f8b2"} Jul 13 17:07:55.599385 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:55.599344 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" event={"ID":"57625ec7-ea64-4acd-a884-68fec13cadbe","Type":"ContainerStarted","Data":"4b28458b15824c1a8b1d663f78c26cf15e862c673fdab2bd830edd4f4802e9eb"} Jul 13 17:07:55.599806 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:55.599410 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" event={"ID":"57625ec7-ea64-4acd-a884-68fec13cadbe","Type":"ContainerStarted","Data":"60be951fa8ea09a4746fd27a8bf6812d0cfb909c5c87ff2348417e141779052a"} Jul 13 17:07:55.625280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:55.625225 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-76dc565d9f-w97hl" podStartSLOduration=1.6256155429999999 podStartE2EDuration="4.625202636s" totalImagesPullingTime="2.999587093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:07:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:07:55.624599483 +0000 UTC m=+153.287374672" watchObservedRunningTime="2026-07-13 17:07:55.625202636 +0000 UTC m=+153.287977823" Jul 13 17:07:56.435968 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.435924 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cd5bf477d-fgkbk"] Jul 13 17:07:56.438233 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.438208 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.451207 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.451131 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cd5bf477d-fgkbk"] Jul 13 17:07:56.510009 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.509923 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-console-config\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.510009 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.509978 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-service-ca\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.510270 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.510066 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/29be27a5-4c8d-4189-989b-69921c3bc408-console-oauth-config\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.510270 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.510134 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/29be27a5-4c8d-4189-989b-69921c3bc408-console-serving-cert\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.510270 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.510158 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-oauth-serving-cert\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.510270 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.510191 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-trusted-ca-bundle\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.510270 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.510229 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq4tr\" (UniqueName: \"kubernetes.io/projected/29be27a5-4c8d-4189-989b-69921c3bc408-kube-api-access-tq4tr\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.556962 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.556927 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:07:56.613277 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.611419 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq4tr\" (UniqueName: \"kubernetes.io/projected/29be27a5-4c8d-4189-989b-69921c3bc408-kube-api-access-tq4tr\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.613277 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.611491 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-console-config\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.613277 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.611529 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-service-ca\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.613277 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.611558 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/29be27a5-4c8d-4189-989b-69921c3bc408-console-oauth-config\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.613277 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.611599 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/29be27a5-4c8d-4189-989b-69921c3bc408-console-serving-cert\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.613277 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.611624 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-oauth-serving-cert\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.613277 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.611659 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-trusted-ca-bundle\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.613277 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.612871 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-trusted-ca-bundle\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.614401 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.613352 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-console-config\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.614401 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.613777 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-service-ca\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.615049 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.615023 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-oauth-serving-cert\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.615183 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.615027 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:07:56.615895 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.615866 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/29be27a5-4c8d-4189-989b-69921c3bc408-console-oauth-config\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.616111 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.616094 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/29be27a5-4c8d-4189-989b-69921c3bc408-console-serving-cert\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.620940 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.620919 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq4tr\" (UniqueName: \"kubernetes.io/projected/29be27a5-4c8d-4189-989b-69921c3bc408-kube-api-access-tq4tr\") pod \"console-6cd5bf477d-fgkbk\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.667961 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.667926 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:07:56.750431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.750342 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:07:56.887409 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:56.887366 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cd5bf477d-fgkbk"] Jul 13 17:07:56.890122 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:07:56.890089 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29be27a5_4c8d_4189_989b_69921c3bc408.slice/crio-0367c5919dfc4f9bc29e422a3c33c2924df178b00c051a481eebca88e8305024 WatchSource:0}: Error finding container 0367c5919dfc4f9bc29e422a3c33c2924df178b00c051a481eebca88e8305024: Status 404 returned error can't find the container with id 0367c5919dfc4f9bc29e422a3c33c2924df178b00c051a481eebca88e8305024 Jul 13 17:07:57.235329 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:57.235275 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:07:57.291746 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:57.288457 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:07:57.608760 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:57.608651 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd5bf477d-fgkbk" event={"ID":"29be27a5-4c8d-4189-989b-69921c3bc408","Type":"ContainerStarted","Data":"b8e47eb35abb30cf7234078a0479dd72e56399cb3bb252462c996dd919f39aca"} Jul 13 17:07:57.608760 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:57.608742 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd5bf477d-fgkbk" event={"ID":"29be27a5-4c8d-4189-989b-69921c3bc408","Type":"ContainerStarted","Data":"0367c5919dfc4f9bc29e422a3c33c2924df178b00c051a481eebca88e8305024"} Jul 13 17:07:57.627001 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:07:57.626947 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cd5bf477d-fgkbk" podStartSLOduration=1.626930398 podStartE2EDuration="1.626930398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:07:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:07:57.626336145 +0000 UTC m=+155.289111333" watchObservedRunningTime="2026-07-13 17:07:57.626930398 +0000 UTC m=+155.289705585" Jul 13 17:08:06.750951 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:06.750906 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:08:06.751335 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:06.750967 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:08:06.755974 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:06.755950 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:08:07.649067 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:07.649037 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:08:07.701988 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:07.701957 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b6997cb96-zqv5g"] Jul 13 17:08:07.828681 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:07.828644 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:08:07.930048 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:07.929958 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:08:08.001462 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:08.001404 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:08:32.724047 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:32.723959 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b6997cb96-zqv5g" podUID="245d8353-8f05-4fe3-86d0-46c66008a8fc" containerName="console" containerID="cri-o://2f9f4b6aefddbf671fda9dd907f2f9e14b43e093d16bdcae8cfcec09633e8bfc" gracePeriod=15 Jul 13 17:08:32.963156 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:32.963129 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:08:33.097885 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.097777 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87h8v\" (UniqueName: \"kubernetes.io/projected/245d8353-8f05-4fe3-86d0-46c66008a8fc-kube-api-access-87h8v\") pod \"245d8353-8f05-4fe3-86d0-46c66008a8fc\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " Jul 13 17:08:33.097885 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.097856 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/245d8353-8f05-4fe3-86d0-46c66008a8fc-console-serving-cert\") pod \"245d8353-8f05-4fe3-86d0-46c66008a8fc\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " Jul 13 17:08:33.098094 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.097945 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-console-config\") pod \"245d8353-8f05-4fe3-86d0-46c66008a8fc\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " Jul 13 17:08:33.098094 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.098023 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/245d8353-8f05-4fe3-86d0-46c66008a8fc-console-oauth-config\") pod \"245d8353-8f05-4fe3-86d0-46c66008a8fc\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " Jul 13 17:08:33.098094 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.098060 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-trusted-ca-bundle\") pod \"245d8353-8f05-4fe3-86d0-46c66008a8fc\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " Jul 13 17:08:33.098246 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.098119 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-service-ca\") pod \"245d8353-8f05-4fe3-86d0-46c66008a8fc\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " Jul 13 17:08:33.098246 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.098159 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-oauth-serving-cert\") pod \"245d8353-8f05-4fe3-86d0-46c66008a8fc\" (UID: \"245d8353-8f05-4fe3-86d0-46c66008a8fc\") " Jul 13 17:08:33.098246 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.098221 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-console-config" (OuterVolumeSpecName: "console-config") pod "245d8353-8f05-4fe3-86d0-46c66008a8fc" (UID: "245d8353-8f05-4fe3-86d0-46c66008a8fc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:08:33.098526 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.098497 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "245d8353-8f05-4fe3-86d0-46c66008a8fc" (UID: "245d8353-8f05-4fe3-86d0-46c66008a8fc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:08:33.098669 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.098552 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "245d8353-8f05-4fe3-86d0-46c66008a8fc" (UID: "245d8353-8f05-4fe3-86d0-46c66008a8fc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:08:33.098669 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.098592 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-service-ca" (OuterVolumeSpecName: "service-ca") pod "245d8353-8f05-4fe3-86d0-46c66008a8fc" (UID: "245d8353-8f05-4fe3-86d0-46c66008a8fc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:08:33.098669 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.098649 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-console-config\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:08:33.100499 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.100468 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/245d8353-8f05-4fe3-86d0-46c66008a8fc-kube-api-access-87h8v" (OuterVolumeSpecName: "kube-api-access-87h8v") pod "245d8353-8f05-4fe3-86d0-46c66008a8fc" (UID: "245d8353-8f05-4fe3-86d0-46c66008a8fc"). InnerVolumeSpecName "kube-api-access-87h8v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:08:33.100499 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.100482 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/245d8353-8f05-4fe3-86d0-46c66008a8fc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "245d8353-8f05-4fe3-86d0-46c66008a8fc" (UID: "245d8353-8f05-4fe3-86d0-46c66008a8fc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:08:33.100681 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.100550 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/245d8353-8f05-4fe3-86d0-46c66008a8fc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "245d8353-8f05-4fe3-86d0-46c66008a8fc" (UID: "245d8353-8f05-4fe3-86d0-46c66008a8fc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:08:33.199443 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.199407 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/245d8353-8f05-4fe3-86d0-46c66008a8fc-console-oauth-config\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:08:33.199443 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.199440 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-trusted-ca-bundle\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:08:33.199443 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.199450 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-service-ca\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:08:33.199691 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.199459 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/245d8353-8f05-4fe3-86d0-46c66008a8fc-oauth-serving-cert\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:08:33.199691 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.199467 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87h8v\" (UniqueName: \"kubernetes.io/projected/245d8353-8f05-4fe3-86d0-46c66008a8fc-kube-api-access-87h8v\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:08:33.199691 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.199476 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/245d8353-8f05-4fe3-86d0-46c66008a8fc-console-serving-cert\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:08:33.734094 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.734057 2564 generic.go:356] "Generic (PLEG): container finished" podID="245d8353-8f05-4fe3-86d0-46c66008a8fc" containerID="2f9f4b6aefddbf671fda9dd907f2f9e14b43e093d16bdcae8cfcec09633e8bfc" exitCode=0 Jul 13 17:08:33.734478 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.734100 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b6997cb96-zqv5g" Jul 13 17:08:33.734478 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.734132 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b6997cb96-zqv5g" event={"ID":"245d8353-8f05-4fe3-86d0-46c66008a8fc","Type":"ContainerDied","Data":"2f9f4b6aefddbf671fda9dd907f2f9e14b43e093d16bdcae8cfcec09633e8bfc"} Jul 13 17:08:33.734478 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.734181 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b6997cb96-zqv5g" event={"ID":"245d8353-8f05-4fe3-86d0-46c66008a8fc","Type":"ContainerDied","Data":"99591dd7597f14172cedd88b8a8a425948cddad077b59d6e5350ccab9d4a0b21"} Jul 13 17:08:33.734478 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.734202 2564 scope.go:117] "RemoveContainer" containerID="2f9f4b6aefddbf671fda9dd907f2f9e14b43e093d16bdcae8cfcec09633e8bfc" Jul 13 17:08:33.742584 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.742568 2564 scope.go:117] "RemoveContainer" containerID="2f9f4b6aefddbf671fda9dd907f2f9e14b43e093d16bdcae8cfcec09633e8bfc" Jul 13 17:08:33.742863 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:08:33.742844 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f9f4b6aefddbf671fda9dd907f2f9e14b43e093d16bdcae8cfcec09633e8bfc\": container with ID starting with 2f9f4b6aefddbf671fda9dd907f2f9e14b43e093d16bdcae8cfcec09633e8bfc not found: ID does not exist" containerID="2f9f4b6aefddbf671fda9dd907f2f9e14b43e093d16bdcae8cfcec09633e8bfc" Jul 13 17:08:33.742931 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.742874 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f9f4b6aefddbf671fda9dd907f2f9e14b43e093d16bdcae8cfcec09633e8bfc"} err="failed to get container status \"2f9f4b6aefddbf671fda9dd907f2f9e14b43e093d16bdcae8cfcec09633e8bfc\": rpc error: code = NotFound desc = could not find container \"2f9f4b6aefddbf671fda9dd907f2f9e14b43e093d16bdcae8cfcec09633e8bfc\": container with ID starting with 2f9f4b6aefddbf671fda9dd907f2f9e14b43e093d16bdcae8cfcec09633e8bfc not found: ID does not exist" Jul 13 17:08:33.754551 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.754531 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b6997cb96-zqv5g"] Jul 13 17:08:33.758007 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:33.757986 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b6997cb96-zqv5g"] Jul 13 17:08:34.959236 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:34.959194 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="245d8353-8f05-4fe3-86d0-46c66008a8fc" path="/var/lib/kubelet/pods/245d8353-8f05-4fe3-86d0-46c66008a8fc/volumes" Jul 13 17:08:39.611952 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:39.611903 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:08:39.673164 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:08:39.673129 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:09:01.370206 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:01.370158 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:09:01.436836 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:01.436799 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:09:13.218814 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:13.218771 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:09:13.270914 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:13.270882 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:09:17.849017 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:17.848982 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5496c587bb-mq8nd"] Jul 13 17:09:17.849401 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:17.849240 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="245d8353-8f05-4fe3-86d0-46c66008a8fc" containerName="console" Jul 13 17:09:17.849401 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:17.849250 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="245d8353-8f05-4fe3-86d0-46c66008a8fc" containerName="console" Jul 13 17:09:17.849401 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:17.849287 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="245d8353-8f05-4fe3-86d0-46c66008a8fc" containerName="console" Jul 13 17:09:17.853324 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:17.853301 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:17.861916 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:17.861888 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5496c587bb-mq8nd"] Jul 13 17:09:17.898236 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:17.898202 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:09:17.918364 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:17.918315 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bac6b64-1ca5-494e-a65d-5507b3b071b2-console-serving-cert\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:17.918531 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:17.918384 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-console-config\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:17.918531 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:17.918417 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-trusted-ca-bundle\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:17.918531 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:17.918464 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k27l9\" (UniqueName: \"kubernetes.io/projected/8bac6b64-1ca5-494e-a65d-5507b3b071b2-kube-api-access-k27l9\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:17.918531 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:17.918486 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-service-ca\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:17.918531 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:17.918508 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bac6b64-1ca5-494e-a65d-5507b3b071b2-console-oauth-config\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:17.918531 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:17.918530 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-oauth-serving-cert\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:17.999192 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:17.999157 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:09:18.019544 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.019510 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k27l9\" (UniqueName: \"kubernetes.io/projected/8bac6b64-1ca5-494e-a65d-5507b3b071b2-kube-api-access-k27l9\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:18.019661 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.019560 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-service-ca\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:18.019661 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.019590 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bac6b64-1ca5-494e-a65d-5507b3b071b2-console-oauth-config\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:18.019791 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.019763 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-oauth-serving-cert\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:18.019869 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.019851 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bac6b64-1ca5-494e-a65d-5507b3b071b2-console-serving-cert\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:18.019927 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.019913 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-console-config\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:18.019976 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.019948 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-trusted-ca-bundle\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:18.020459 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.020433 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-service-ca\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:18.020583 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.020433 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-oauth-serving-cert\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:18.020697 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.020677 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-trusted-ca-bundle\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:18.020808 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.020786 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-console-config\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:18.022171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.022141 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bac6b64-1ca5-494e-a65d-5507b3b071b2-console-oauth-config\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:18.022359 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.022339 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bac6b64-1ca5-494e-a65d-5507b3b071b2-console-serving-cert\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:18.029398 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.029378 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k27l9\" (UniqueName: \"kubernetes.io/projected/8bac6b64-1ca5-494e-a65d-5507b3b071b2-kube-api-access-k27l9\") pod \"console-5496c587bb-mq8nd\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:18.122127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.122033 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:09:18.163588 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.163548 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:18.201521 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.201485 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:09:18.298458 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.298423 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5496c587bb-mq8nd"] Jul 13 17:09:18.301833 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:09:18.301806 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bac6b64_1ca5_494e_a65d_5507b3b071b2.slice/crio-32eed5036f73203e51026ccc5d378f366774101c3f047d2f165f29ca35326328 WatchSource:0}: Error finding container 32eed5036f73203e51026ccc5d378f366774101c3f047d2f165f29ca35326328: Status 404 returned error can't find the container with id 32eed5036f73203e51026ccc5d378f366774101c3f047d2f165f29ca35326328 Jul 13 17:09:18.888509 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.888471 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5496c587bb-mq8nd" event={"ID":"8bac6b64-1ca5-494e-a65d-5507b3b071b2","Type":"ContainerStarted","Data":"f0eafa474e9d4c387dbd94463fcde4af3230735f3033cd96cb19e981445be2c5"} Jul 13 17:09:18.889002 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.888522 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5496c587bb-mq8nd" event={"ID":"8bac6b64-1ca5-494e-a65d-5507b3b071b2","Type":"ContainerStarted","Data":"32eed5036f73203e51026ccc5d378f366774101c3f047d2f165f29ca35326328"} Jul 13 17:09:18.907040 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:18.906983 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5496c587bb-mq8nd" podStartSLOduration=1.906967428 podStartE2EDuration="1.906967428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:09:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:09:18.906023553 +0000 UTC m=+236.568798741" watchObservedRunningTime="2026-07-13 17:09:18.906967428 +0000 UTC m=+236.569742687" Jul 13 17:09:28.163813 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:28.163758 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:28.164261 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:28.163837 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:28.168653 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:28.168626 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:28.926796 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:28.926768 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:09:28.970941 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:28.970909 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cd5bf477d-fgkbk"] Jul 13 17:09:29.077226 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:29.077185 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:09:29.178449 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:29.178358 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:09:29.236233 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:29.236204 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:09:32.865939 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:32.865887 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:09:32.966789 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:32.966753 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:09:53.997687 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:53.997620 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6cd5bf477d-fgkbk" podUID="29be27a5-4c8d-4189-989b-69921c3bc408" containerName="console" containerID="cri-o://b8e47eb35abb30cf7234078a0479dd72e56399cb3bb252462c996dd919f39aca" gracePeriod=15 Jul 13 17:09:54.233416 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.233388 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:09:54.402938 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.402913 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-oauth-serving-cert\") pod \"29be27a5-4c8d-4189-989b-69921c3bc408\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " Jul 13 17:09:54.403097 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.402966 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tq4tr\" (UniqueName: \"kubernetes.io/projected/29be27a5-4c8d-4189-989b-69921c3bc408-kube-api-access-tq4tr\") pod \"29be27a5-4c8d-4189-989b-69921c3bc408\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " Jul 13 17:09:54.403097 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.402991 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-service-ca\") pod \"29be27a5-4c8d-4189-989b-69921c3bc408\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " Jul 13 17:09:54.403097 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.403012 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/29be27a5-4c8d-4189-989b-69921c3bc408-console-oauth-config\") pod \"29be27a5-4c8d-4189-989b-69921c3bc408\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " Jul 13 17:09:54.403097 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.403066 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/29be27a5-4c8d-4189-989b-69921c3bc408-console-serving-cert\") pod \"29be27a5-4c8d-4189-989b-69921c3bc408\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " Jul 13 17:09:54.403270 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.403102 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-trusted-ca-bundle\") pod \"29be27a5-4c8d-4189-989b-69921c3bc408\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " Jul 13 17:09:54.403270 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.403148 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-console-config\") pod \"29be27a5-4c8d-4189-989b-69921c3bc408\" (UID: \"29be27a5-4c8d-4189-989b-69921c3bc408\") " Jul 13 17:09:54.403514 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.403482 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "29be27a5-4c8d-4189-989b-69921c3bc408" (UID: "29be27a5-4c8d-4189-989b-69921c3bc408"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:09:54.403639 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.403395 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-service-ca" (OuterVolumeSpecName: "service-ca") pod "29be27a5-4c8d-4189-989b-69921c3bc408" (UID: "29be27a5-4c8d-4189-989b-69921c3bc408"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:09:54.403639 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.403582 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-console-config" (OuterVolumeSpecName: "console-config") pod "29be27a5-4c8d-4189-989b-69921c3bc408" (UID: "29be27a5-4c8d-4189-989b-69921c3bc408"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:09:54.403639 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.403593 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "29be27a5-4c8d-4189-989b-69921c3bc408" (UID: "29be27a5-4c8d-4189-989b-69921c3bc408"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:09:54.405285 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.405262 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29be27a5-4c8d-4189-989b-69921c3bc408-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "29be27a5-4c8d-4189-989b-69921c3bc408" (UID: "29be27a5-4c8d-4189-989b-69921c3bc408"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:09:54.405743 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.405701 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29be27a5-4c8d-4189-989b-69921c3bc408-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "29be27a5-4c8d-4189-989b-69921c3bc408" (UID: "29be27a5-4c8d-4189-989b-69921c3bc408"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:09:54.405804 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.405708 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29be27a5-4c8d-4189-989b-69921c3bc408-kube-api-access-tq4tr" (OuterVolumeSpecName: "kube-api-access-tq4tr") pod "29be27a5-4c8d-4189-989b-69921c3bc408" (UID: "29be27a5-4c8d-4189-989b-69921c3bc408"). InnerVolumeSpecName "kube-api-access-tq4tr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:09:54.504041 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.504006 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/29be27a5-4c8d-4189-989b-69921c3bc408-console-serving-cert\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:09:54.504041 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.504034 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-trusted-ca-bundle\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:09:54.504041 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.504044 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-console-config\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:09:54.504041 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.504053 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-oauth-serving-cert\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:09:54.504281 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.504063 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tq4tr\" (UniqueName: \"kubernetes.io/projected/29be27a5-4c8d-4189-989b-69921c3bc408-kube-api-access-tq4tr\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:09:54.504281 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.504072 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/29be27a5-4c8d-4189-989b-69921c3bc408-service-ca\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:09:54.504281 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:54.504102 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/29be27a5-4c8d-4189-989b-69921c3bc408-console-oauth-config\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:09:55.011153 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:55.011121 2564 generic.go:356] "Generic (PLEG): container finished" podID="29be27a5-4c8d-4189-989b-69921c3bc408" containerID="b8e47eb35abb30cf7234078a0479dd72e56399cb3bb252462c996dd919f39aca" exitCode=0 Jul 13 17:09:55.011559 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:55.011172 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd5bf477d-fgkbk" event={"ID":"29be27a5-4c8d-4189-989b-69921c3bc408","Type":"ContainerDied","Data":"b8e47eb35abb30cf7234078a0479dd72e56399cb3bb252462c996dd919f39aca"} Jul 13 17:09:55.011559 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:55.011176 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd5bf477d-fgkbk" Jul 13 17:09:55.011559 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:55.011212 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd5bf477d-fgkbk" event={"ID":"29be27a5-4c8d-4189-989b-69921c3bc408","Type":"ContainerDied","Data":"0367c5919dfc4f9bc29e422a3c33c2924df178b00c051a481eebca88e8305024"} Jul 13 17:09:55.011559 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:55.011231 2564 scope.go:117] "RemoveContainer" containerID="b8e47eb35abb30cf7234078a0479dd72e56399cb3bb252462c996dd919f39aca" Jul 13 17:09:55.019156 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:55.019137 2564 scope.go:117] "RemoveContainer" containerID="b8e47eb35abb30cf7234078a0479dd72e56399cb3bb252462c996dd919f39aca" Jul 13 17:09:55.019447 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:09:55.019429 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8e47eb35abb30cf7234078a0479dd72e56399cb3bb252462c996dd919f39aca\": container with ID starting with b8e47eb35abb30cf7234078a0479dd72e56399cb3bb252462c996dd919f39aca not found: ID does not exist" containerID="b8e47eb35abb30cf7234078a0479dd72e56399cb3bb252462c996dd919f39aca" Jul 13 17:09:55.019503 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:55.019455 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8e47eb35abb30cf7234078a0479dd72e56399cb3bb252462c996dd919f39aca"} err="failed to get container status \"b8e47eb35abb30cf7234078a0479dd72e56399cb3bb252462c996dd919f39aca\": rpc error: code = NotFound desc = could not find container \"b8e47eb35abb30cf7234078a0479dd72e56399cb3bb252462c996dd919f39aca\": container with ID starting with b8e47eb35abb30cf7234078a0479dd72e56399cb3bb252462c996dd919f39aca not found: ID does not exist" Jul 13 17:09:55.028608 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:55.028582 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cd5bf477d-fgkbk"] Jul 13 17:09:55.030142 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:55.030120 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6cd5bf477d-fgkbk"] Jul 13 17:09:56.959801 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:09:56.959767 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29be27a5-4c8d-4189-989b-69921c3bc408" path="/var/lib/kubelet/pods/29be27a5-4c8d-4189-989b-69921c3bc408/volumes" Jul 13 17:10:22.831310 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:22.831277 2564 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 17:10:27.016487 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.016450 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:10:27.705419 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.705385 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m"] Jul 13 17:10:27.705643 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.705632 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29be27a5-4c8d-4189-989b-69921c3bc408" containerName="console" Jul 13 17:10:27.705685 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.705645 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="29be27a5-4c8d-4189-989b-69921c3bc408" containerName="console" Jul 13 17:10:27.705718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.705692 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="29be27a5-4c8d-4189-989b-69921c3bc408" containerName="console" Jul 13 17:10:27.708460 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.708443 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m" Jul 13 17:10:27.711823 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.711789 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 17:10:27.711944 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.711877 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 17:10:27.713116 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.713099 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5gvvb\"" Jul 13 17:10:27.718092 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.718071 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m"] Jul 13 17:10:27.730664 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.730642 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2adb50ff-2fd8-41ff-ad56-9df473c00c34-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m\" (UID: \"2adb50ff-2fd8-41ff-ad56-9df473c00c34\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m" Jul 13 17:10:27.730790 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.730680 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2adb50ff-2fd8-41ff-ad56-9df473c00c34-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m\" (UID: \"2adb50ff-2fd8-41ff-ad56-9df473c00c34\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m" Jul 13 17:10:27.730835 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.730781 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xsfg\" (UniqueName: \"kubernetes.io/projected/2adb50ff-2fd8-41ff-ad56-9df473c00c34-kube-api-access-4xsfg\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m\" (UID: \"2adb50ff-2fd8-41ff-ad56-9df473c00c34\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m" Jul 13 17:10:27.831918 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.831882 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2adb50ff-2fd8-41ff-ad56-9df473c00c34-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m\" (UID: \"2adb50ff-2fd8-41ff-ad56-9df473c00c34\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m" Jul 13 17:10:27.832027 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.831942 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xsfg\" (UniqueName: \"kubernetes.io/projected/2adb50ff-2fd8-41ff-ad56-9df473c00c34-kube-api-access-4xsfg\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m\" (UID: \"2adb50ff-2fd8-41ff-ad56-9df473c00c34\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m" Jul 13 17:10:27.832027 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.831978 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2adb50ff-2fd8-41ff-ad56-9df473c00c34-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m\" (UID: \"2adb50ff-2fd8-41ff-ad56-9df473c00c34\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m" Jul 13 17:10:27.832330 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.832310 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2adb50ff-2fd8-41ff-ad56-9df473c00c34-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m\" (UID: \"2adb50ff-2fd8-41ff-ad56-9df473c00c34\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m" Jul 13 17:10:27.832371 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.832353 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2adb50ff-2fd8-41ff-ad56-9df473c00c34-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m\" (UID: \"2adb50ff-2fd8-41ff-ad56-9df473c00c34\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m" Jul 13 17:10:27.841024 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:27.841003 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xsfg\" (UniqueName: \"kubernetes.io/projected/2adb50ff-2fd8-41ff-ad56-9df473c00c34-kube-api-access-4xsfg\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m\" (UID: \"2adb50ff-2fd8-41ff-ad56-9df473c00c34\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m" Jul 13 17:10:28.018268 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:28.018181 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m" Jul 13 17:10:28.151498 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:28.151463 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m"] Jul 13 17:10:28.155752 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:10:28.155705 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2adb50ff_2fd8_41ff_ad56_9df473c00c34.slice/crio-29276ff81d4e6ef2516ae0066c9f2b5efca74a21613a104f67f70d66aa53ddc1 WatchSource:0}: Error finding container 29276ff81d4e6ef2516ae0066c9f2b5efca74a21613a104f67f70d66aa53ddc1: Status 404 returned error can't find the container with id 29276ff81d4e6ef2516ae0066c9f2b5efca74a21613a104f67f70d66aa53ddc1 Jul 13 17:10:28.157671 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:28.157641 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 17:10:28.616839 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:28.616792 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:10:29.129091 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:29.129044 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m" event={"ID":"2adb50ff-2fd8-41ff-ad56-9df473c00c34","Type":"ContainerStarted","Data":"29276ff81d4e6ef2516ae0066c9f2b5efca74a21613a104f67f70d66aa53ddc1"} Jul 13 17:10:35.152501 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:35.152463 2564 generic.go:356] "Generic (PLEG): container finished" podID="2adb50ff-2fd8-41ff-ad56-9df473c00c34" containerID="2ea117e59ad32f3b5870a543cbc7ddd6eedee35b0b3b2f0b0cea5303b5c0d942" exitCode=0 Jul 13 17:10:35.153002 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:35.152555 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m" event={"ID":"2adb50ff-2fd8-41ff-ad56-9df473c00c34","Type":"ContainerDied","Data":"2ea117e59ad32f3b5870a543cbc7ddd6eedee35b0b3b2f0b0cea5303b5c0d942"} Jul 13 17:10:38.165152 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:38.165113 2564 generic.go:356] "Generic (PLEG): container finished" podID="2adb50ff-2fd8-41ff-ad56-9df473c00c34" containerID="b99a6fb15a2603e63b8ba5cde733af199e27209b94efb5e5d402da1db68dbdd0" exitCode=0 Jul 13 17:10:38.165628 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:38.165205 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m" event={"ID":"2adb50ff-2fd8-41ff-ad56-9df473c00c34","Type":"ContainerDied","Data":"b99a6fb15a2603e63b8ba5cde733af199e27209b94efb5e5d402da1db68dbdd0"} Jul 13 17:10:48.202952 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:48.202867 2564 generic.go:356] "Generic (PLEG): container finished" podID="2adb50ff-2fd8-41ff-ad56-9df473c00c34" containerID="54349c978fe3cc6a2d6ee1749b82685336557b1a2d5675038fb7050d37768b58" exitCode=0 Jul 13 17:10:48.202952 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:48.202916 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m" event={"ID":"2adb50ff-2fd8-41ff-ad56-9df473c00c34","Type":"ContainerDied","Data":"54349c978fe3cc6a2d6ee1749b82685336557b1a2d5675038fb7050d37768b58"} Jul 13 17:10:49.322628 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:49.322600 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m" Jul 13 17:10:49.415479 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:49.415441 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2adb50ff-2fd8-41ff-ad56-9df473c00c34-util\") pod \"2adb50ff-2fd8-41ff-ad56-9df473c00c34\" (UID: \"2adb50ff-2fd8-41ff-ad56-9df473c00c34\") " Jul 13 17:10:49.415650 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:49.415530 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xsfg\" (UniqueName: \"kubernetes.io/projected/2adb50ff-2fd8-41ff-ad56-9df473c00c34-kube-api-access-4xsfg\") pod \"2adb50ff-2fd8-41ff-ad56-9df473c00c34\" (UID: \"2adb50ff-2fd8-41ff-ad56-9df473c00c34\") " Jul 13 17:10:49.415650 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:49.415561 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2adb50ff-2fd8-41ff-ad56-9df473c00c34-bundle\") pod \"2adb50ff-2fd8-41ff-ad56-9df473c00c34\" (UID: \"2adb50ff-2fd8-41ff-ad56-9df473c00c34\") " Jul 13 17:10:49.416313 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:49.416279 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2adb50ff-2fd8-41ff-ad56-9df473c00c34-bundle" (OuterVolumeSpecName: "bundle") pod "2adb50ff-2fd8-41ff-ad56-9df473c00c34" (UID: "2adb50ff-2fd8-41ff-ad56-9df473c00c34"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:10:49.417926 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:49.417901 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2adb50ff-2fd8-41ff-ad56-9df473c00c34-kube-api-access-4xsfg" (OuterVolumeSpecName: "kube-api-access-4xsfg") pod "2adb50ff-2fd8-41ff-ad56-9df473c00c34" (UID: "2adb50ff-2fd8-41ff-ad56-9df473c00c34"). InnerVolumeSpecName "kube-api-access-4xsfg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:10:49.419985 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:49.419959 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2adb50ff-2fd8-41ff-ad56-9df473c00c34-util" (OuterVolumeSpecName: "util") pod "2adb50ff-2fd8-41ff-ad56-9df473c00c34" (UID: "2adb50ff-2fd8-41ff-ad56-9df473c00c34"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:10:49.516689 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:49.516595 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xsfg\" (UniqueName: \"kubernetes.io/projected/2adb50ff-2fd8-41ff-ad56-9df473c00c34-kube-api-access-4xsfg\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:10:49.516689 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:49.516627 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2adb50ff-2fd8-41ff-ad56-9df473c00c34-bundle\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:10:49.516689 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:49.516639 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2adb50ff-2fd8-41ff-ad56-9df473c00c34-util\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:10:50.211022 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:50.210989 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m" Jul 13 17:10:50.211022 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:50.211005 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncp6m" event={"ID":"2adb50ff-2fd8-41ff-ad56-9df473c00c34","Type":"ContainerDied","Data":"29276ff81d4e6ef2516ae0066c9f2b5efca74a21613a104f67f70d66aa53ddc1"} Jul 13 17:10:50.211224 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:50.211072 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29276ff81d4e6ef2516ae0066c9f2b5efca74a21613a104f67f70d66aa53ddc1" Jul 13 17:10:53.744082 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:53.744042 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:10:54.602821 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:54.602786 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:10:55.687687 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.687653 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p87l5"] Jul 13 17:10:55.688084 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.687907 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2adb50ff-2fd8-41ff-ad56-9df473c00c34" containerName="util" Jul 13 17:10:55.688084 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.687917 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2adb50ff-2fd8-41ff-ad56-9df473c00c34" containerName="util" Jul 13 17:10:55.688084 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.687925 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2adb50ff-2fd8-41ff-ad56-9df473c00c34" containerName="extract" Jul 13 17:10:55.688084 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.687931 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2adb50ff-2fd8-41ff-ad56-9df473c00c34" containerName="extract" Jul 13 17:10:55.688084 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.687943 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2adb50ff-2fd8-41ff-ad56-9df473c00c34" containerName="pull" Jul 13 17:10:55.688084 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.687948 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2adb50ff-2fd8-41ff-ad56-9df473c00c34" containerName="pull" Jul 13 17:10:55.688084 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.687992 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2adb50ff-2fd8-41ff-ad56-9df473c00c34" containerName="extract" Jul 13 17:10:55.691676 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.691658 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p87l5" Jul 13 17:10:55.696664 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.696638 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 13 17:10:55.696840 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.696815 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 13 17:10:55.696974 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.696849 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-b68hw\"" Jul 13 17:10:55.704692 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.704667 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p87l5"] Jul 13 17:10:55.761441 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.761402 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw427\" (UniqueName: \"kubernetes.io/projected/4e7c3374-5752-40da-82b7-4950ed78536d-kube-api-access-rw427\") pod \"cert-manager-operator-controller-manager-55c444bf86-p87l5\" (UID: \"4e7c3374-5752-40da-82b7-4950ed78536d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p87l5" Jul 13 17:10:55.761616 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.761466 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e7c3374-5752-40da-82b7-4950ed78536d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-p87l5\" (UID: \"4e7c3374-5752-40da-82b7-4950ed78536d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p87l5" Jul 13 17:10:55.862003 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.861970 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e7c3374-5752-40da-82b7-4950ed78536d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-p87l5\" (UID: \"4e7c3374-5752-40da-82b7-4950ed78536d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p87l5" Jul 13 17:10:55.862142 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.862033 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw427\" (UniqueName: \"kubernetes.io/projected/4e7c3374-5752-40da-82b7-4950ed78536d-kube-api-access-rw427\") pod \"cert-manager-operator-controller-manager-55c444bf86-p87l5\" (UID: \"4e7c3374-5752-40da-82b7-4950ed78536d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p87l5" Jul 13 17:10:55.862387 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.862363 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e7c3374-5752-40da-82b7-4950ed78536d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-p87l5\" (UID: \"4e7c3374-5752-40da-82b7-4950ed78536d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p87l5" Jul 13 17:10:55.875143 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:55.875112 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw427\" (UniqueName: \"kubernetes.io/projected/4e7c3374-5752-40da-82b7-4950ed78536d-kube-api-access-rw427\") pod \"cert-manager-operator-controller-manager-55c444bf86-p87l5\" (UID: \"4e7c3374-5752-40da-82b7-4950ed78536d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p87l5" Jul 13 17:10:56.001295 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:56.001204 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p87l5" Jul 13 17:10:56.140162 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:56.140133 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p87l5"] Jul 13 17:10:56.142603 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:10:56.142566 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e7c3374_5752_40da_82b7_4950ed78536d.slice/crio-ea1e517651f06f216e41432ed5f7d13756d10c356af4ce4134ad1fff4e538981 WatchSource:0}: Error finding container ea1e517651f06f216e41432ed5f7d13756d10c356af4ce4134ad1fff4e538981: Status 404 returned error can't find the container with id ea1e517651f06f216e41432ed5f7d13756d10c356af4ce4134ad1fff4e538981 Jul 13 17:10:56.232510 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:56.232477 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p87l5" event={"ID":"4e7c3374-5752-40da-82b7-4950ed78536d","Type":"ContainerStarted","Data":"ea1e517651f06f216e41432ed5f7d13756d10c356af4ce4134ad1fff4e538981"} Jul 13 17:10:57.347889 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:57.347856 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:10:57.448911 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:57.448866 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:10:58.242773 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:58.242714 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p87l5" event={"ID":"4e7c3374-5752-40da-82b7-4950ed78536d","Type":"ContainerStarted","Data":"a6be3e6a5cd1ea89bbd91dae04882bf1b1bf059b19389622b10851037a9a8500"} Jul 13 17:10:58.274902 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:58.274831 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p87l5" podStartSLOduration=1.657965307 podStartE2EDuration="3.274804813s" totalImagesPullingTime="1.616839506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:10:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:10:58.273161444 +0000 UTC m=+335.935936633" watchObservedRunningTime="2026-07-13 17:10:58.274804813 +0000 UTC m=+335.937580002" Jul 13 17:10:58.667012 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:58.666986 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:10:58.799892 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:58.799857 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:10:59.373902 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.373863 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8"] Jul 13 17:10:59.376306 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.376288 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8" Jul 13 17:10:59.380234 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.380213 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 17:10:59.381849 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.381826 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 17:10:59.382001 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.381834 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5gvvb\"" Jul 13 17:10:59.394090 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.394066 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8"] Jul 13 17:10:59.481151 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.481113 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:10:59.493813 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.493781 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l66d9\" (UniqueName: \"kubernetes.io/projected/9a162f09-5850-433e-bb8f-7e05f72c268d-kube-api-access-l66d9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8\" (UID: \"9a162f09-5850-433e-bb8f-7e05f72c268d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8" Jul 13 17:10:59.493961 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.493833 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a162f09-5850-433e-bb8f-7e05f72c268d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8\" (UID: \"9a162f09-5850-433e-bb8f-7e05f72c268d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8" Jul 13 17:10:59.493961 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.493854 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a162f09-5850-433e-bb8f-7e05f72c268d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8\" (UID: \"9a162f09-5850-433e-bb8f-7e05f72c268d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8" Jul 13 17:10:59.595156 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.595115 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l66d9\" (UniqueName: \"kubernetes.io/projected/9a162f09-5850-433e-bb8f-7e05f72c268d-kube-api-access-l66d9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8\" (UID: \"9a162f09-5850-433e-bb8f-7e05f72c268d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8" Jul 13 17:10:59.595333 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.595185 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a162f09-5850-433e-bb8f-7e05f72c268d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8\" (UID: \"9a162f09-5850-433e-bb8f-7e05f72c268d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8" Jul 13 17:10:59.595333 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.595297 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a162f09-5850-433e-bb8f-7e05f72c268d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8\" (UID: \"9a162f09-5850-433e-bb8f-7e05f72c268d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8" Jul 13 17:10:59.595539 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.595519 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a162f09-5850-433e-bb8f-7e05f72c268d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8\" (UID: \"9a162f09-5850-433e-bb8f-7e05f72c268d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8" Jul 13 17:10:59.595641 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.595621 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a162f09-5850-433e-bb8f-7e05f72c268d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8\" (UID: \"9a162f09-5850-433e-bb8f-7e05f72c268d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8" Jul 13 17:10:59.612638 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.612602 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l66d9\" (UniqueName: \"kubernetes.io/projected/9a162f09-5850-433e-bb8f-7e05f72c268d-kube-api-access-l66d9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8\" (UID: \"9a162f09-5850-433e-bb8f-7e05f72c268d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8" Jul 13 17:10:59.641035 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.640940 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:10:59.685747 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.685703 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8" Jul 13 17:10:59.818373 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:10:59.818338 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8"] Jul 13 17:10:59.821167 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:10:59.821140 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a162f09_5850_433e_bb8f_7e05f72c268d.slice/crio-7ea6688d4e0279f5ca532606eeb658c3141f9e555230aa699e0d1c0e9cc7b3eb WatchSource:0}: Error finding container 7ea6688d4e0279f5ca532606eeb658c3141f9e555230aa699e0d1c0e9cc7b3eb: Status 404 returned error can't find the container with id 7ea6688d4e0279f5ca532606eeb658c3141f9e555230aa699e0d1c0e9cc7b3eb Jul 13 17:11:00.252487 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:00.252397 2564 generic.go:356] "Generic (PLEG): container finished" podID="9a162f09-5850-433e-bb8f-7e05f72c268d" containerID="f0a9224a058fcef78fc9d87039482ba4baaca939573af0087562cba5a325c070" exitCode=0 Jul 13 17:11:00.252640 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:00.252480 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8" event={"ID":"9a162f09-5850-433e-bb8f-7e05f72c268d","Type":"ContainerDied","Data":"f0a9224a058fcef78fc9d87039482ba4baaca939573af0087562cba5a325c070"} Jul 13 17:11:00.252640 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:00.252527 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8" event={"ID":"9a162f09-5850-433e-bb8f-7e05f72c268d","Type":"ContainerStarted","Data":"7ea6688d4e0279f5ca532606eeb658c3141f9e555230aa699e0d1c0e9cc7b3eb"} Jul 13 17:11:01.636380 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:01.636329 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:02.041463 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.041429 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:02.146079 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.146049 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kqszj"] Jul 13 17:11:02.153089 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.153063 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-kqszj" Jul 13 17:11:02.157537 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.157274 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 13 17:11:02.157537 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.157292 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-sg2g4\"" Jul 13 17:11:02.157537 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.157304 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 13 17:11:02.158014 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.157987 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kqszj"] Jul 13 17:11:02.164859 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.164834 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-j5wdn"] Jul 13 17:11:02.167237 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.167217 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j5wdn" Jul 13 17:11:02.170114 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.170090 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-g897k\"" Jul 13 17:11:02.179653 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.179623 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-j5wdn"] Jul 13 17:11:02.319157 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.319062 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wtm4\" (UniqueName: \"kubernetes.io/projected/2172fe62-0b80-4f9c-8820-dd3497b23640-kube-api-access-6wtm4\") pod \"cert-manager-webhook-6486ffd997-kqszj\" (UID: \"2172fe62-0b80-4f9c-8820-dd3497b23640\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kqszj" Jul 13 17:11:02.319157 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.319141 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2172fe62-0b80-4f9c-8820-dd3497b23640-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kqszj\" (UID: \"2172fe62-0b80-4f9c-8820-dd3497b23640\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kqszj" Jul 13 17:11:02.319376 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.319191 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5bsv\" (UniqueName: \"kubernetes.io/projected/a52fbba2-b14a-4d86-87ae-f45db2770d7e-kube-api-access-t5bsv\") pod \"cert-manager-cainjector-559cdd8c87-j5wdn\" (UID: \"a52fbba2-b14a-4d86-87ae-f45db2770d7e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j5wdn" Jul 13 17:11:02.319376 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.319227 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a52fbba2-b14a-4d86-87ae-f45db2770d7e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-j5wdn\" (UID: \"a52fbba2-b14a-4d86-87ae-f45db2770d7e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j5wdn" Jul 13 17:11:02.420401 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.420365 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a52fbba2-b14a-4d86-87ae-f45db2770d7e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-j5wdn\" (UID: \"a52fbba2-b14a-4d86-87ae-f45db2770d7e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j5wdn" Jul 13 17:11:02.420613 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.420439 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wtm4\" (UniqueName: \"kubernetes.io/projected/2172fe62-0b80-4f9c-8820-dd3497b23640-kube-api-access-6wtm4\") pod \"cert-manager-webhook-6486ffd997-kqszj\" (UID: \"2172fe62-0b80-4f9c-8820-dd3497b23640\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kqszj" Jul 13 17:11:02.420613 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.420502 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2172fe62-0b80-4f9c-8820-dd3497b23640-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kqszj\" (UID: \"2172fe62-0b80-4f9c-8820-dd3497b23640\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kqszj" Jul 13 17:11:02.420613 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.420540 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5bsv\" (UniqueName: \"kubernetes.io/projected/a52fbba2-b14a-4d86-87ae-f45db2770d7e-kube-api-access-t5bsv\") pod \"cert-manager-cainjector-559cdd8c87-j5wdn\" (UID: \"a52fbba2-b14a-4d86-87ae-f45db2770d7e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j5wdn" Jul 13 17:11:02.429750 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.429698 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a52fbba2-b14a-4d86-87ae-f45db2770d7e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-j5wdn\" (UID: \"a52fbba2-b14a-4d86-87ae-f45db2770d7e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j5wdn" Jul 13 17:11:02.429923 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.429698 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2172fe62-0b80-4f9c-8820-dd3497b23640-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kqszj\" (UID: \"2172fe62-0b80-4f9c-8820-dd3497b23640\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kqszj" Jul 13 17:11:02.429923 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.429897 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wtm4\" (UniqueName: \"kubernetes.io/projected/2172fe62-0b80-4f9c-8820-dd3497b23640-kube-api-access-6wtm4\") pod \"cert-manager-webhook-6486ffd997-kqszj\" (UID: \"2172fe62-0b80-4f9c-8820-dd3497b23640\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kqszj" Jul 13 17:11:02.430012 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.429923 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5bsv\" (UniqueName: \"kubernetes.io/projected/a52fbba2-b14a-4d86-87ae-f45db2770d7e-kube-api-access-t5bsv\") pod \"cert-manager-cainjector-559cdd8c87-j5wdn\" (UID: \"a52fbba2-b14a-4d86-87ae-f45db2770d7e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j5wdn" Jul 13 17:11:02.480874 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.480822 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-kqszj" Jul 13 17:11:02.490487 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.490459 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j5wdn" Jul 13 17:11:02.668452 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.668417 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-j5wdn"] Jul 13 17:11:02.671101 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:11:02.671061 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda52fbba2_b14a_4d86_87ae_f45db2770d7e.slice/crio-ee4ecf5404c7e528b1eb3a42e7867440156dc3d1976b21ce18395bbce0bafcc3 WatchSource:0}: Error finding container ee4ecf5404c7e528b1eb3a42e7867440156dc3d1976b21ce18395bbce0bafcc3: Status 404 returned error can't find the container with id ee4ecf5404c7e528b1eb3a42e7867440156dc3d1976b21ce18395bbce0bafcc3 Jul 13 17:11:02.686034 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:02.686004 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kqszj"] Jul 13 17:11:02.688829 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:11:02.688803 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2172fe62_0b80_4f9c_8820_dd3497b23640.slice/crio-9b0dfac9642eb937a1b8d09e17696ee5fbded35655a8842ae1f1f063c7d401cf WatchSource:0}: Error finding container 9b0dfac9642eb937a1b8d09e17696ee5fbded35655a8842ae1f1f063c7d401cf: Status 404 returned error can't find the container with id 9b0dfac9642eb937a1b8d09e17696ee5fbded35655a8842ae1f1f063c7d401cf Jul 13 17:11:03.270064 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:03.270025 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j5wdn" event={"ID":"a52fbba2-b14a-4d86-87ae-f45db2770d7e","Type":"ContainerStarted","Data":"ee4ecf5404c7e528b1eb3a42e7867440156dc3d1976b21ce18395bbce0bafcc3"} Jul 13 17:11:03.271786 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:03.271757 2564 generic.go:356] "Generic (PLEG): container finished" podID="9a162f09-5850-433e-bb8f-7e05f72c268d" containerID="58c62eda032b731ca1eb9801f46883e8bafe7e42601ea8f6654697cd1d588a1e" exitCode=0 Jul 13 17:11:03.271907 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:03.271837 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8" event={"ID":"9a162f09-5850-433e-bb8f-7e05f72c268d","Type":"ContainerDied","Data":"58c62eda032b731ca1eb9801f46883e8bafe7e42601ea8f6654697cd1d588a1e"} Jul 13 17:11:03.273105 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:03.273081 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-kqszj" event={"ID":"2172fe62-0b80-4f9c-8820-dd3497b23640","Type":"ContainerStarted","Data":"9b0dfac9642eb937a1b8d09e17696ee5fbded35655a8842ae1f1f063c7d401cf"} Jul 13 17:11:04.171559 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:04.171521 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:04.252807 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:04.252762 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:04.281783 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:04.281464 2564 generic.go:356] "Generic (PLEG): container finished" podID="9a162f09-5850-433e-bb8f-7e05f72c268d" containerID="2e8318dcfe008f90ac0fd0a6841ffaab3b79ffa2c0ee78eb07247186f891734b" exitCode=0 Jul 13 17:11:04.281783 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:04.281587 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8" event={"ID":"9a162f09-5850-433e-bb8f-7e05f72c268d","Type":"ContainerDied","Data":"2e8318dcfe008f90ac0fd0a6841ffaab3b79ffa2c0ee78eb07247186f891734b"} Jul 13 17:11:05.506574 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:05.506549 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8" Jul 13 17:11:05.646749 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:05.646684 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a162f09-5850-433e-bb8f-7e05f72c268d-bundle\") pod \"9a162f09-5850-433e-bb8f-7e05f72c268d\" (UID: \"9a162f09-5850-433e-bb8f-7e05f72c268d\") " Jul 13 17:11:05.646935 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:05.646753 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l66d9\" (UniqueName: \"kubernetes.io/projected/9a162f09-5850-433e-bb8f-7e05f72c268d-kube-api-access-l66d9\") pod \"9a162f09-5850-433e-bb8f-7e05f72c268d\" (UID: \"9a162f09-5850-433e-bb8f-7e05f72c268d\") " Jul 13 17:11:05.646935 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:05.646794 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a162f09-5850-433e-bb8f-7e05f72c268d-util\") pod \"9a162f09-5850-433e-bb8f-7e05f72c268d\" (UID: \"9a162f09-5850-433e-bb8f-7e05f72c268d\") " Jul 13 17:11:05.647380 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:05.647347 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a162f09-5850-433e-bb8f-7e05f72c268d-bundle" (OuterVolumeSpecName: "bundle") pod "9a162f09-5850-433e-bb8f-7e05f72c268d" (UID: "9a162f09-5850-433e-bb8f-7e05f72c268d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:11:05.649291 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:05.649267 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a162f09-5850-433e-bb8f-7e05f72c268d-kube-api-access-l66d9" (OuterVolumeSpecName: "kube-api-access-l66d9") pod "9a162f09-5850-433e-bb8f-7e05f72c268d" (UID: "9a162f09-5850-433e-bb8f-7e05f72c268d"). InnerVolumeSpecName "kube-api-access-l66d9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:11:05.651495 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:05.651464 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a162f09-5850-433e-bb8f-7e05f72c268d-util" (OuterVolumeSpecName: "util") pod "9a162f09-5850-433e-bb8f-7e05f72c268d" (UID: "9a162f09-5850-433e-bb8f-7e05f72c268d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:11:05.748262 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:05.748201 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a162f09-5850-433e-bb8f-7e05f72c268d-util\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:11:05.748262 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:05.748255 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a162f09-5850-433e-bb8f-7e05f72c268d-bundle\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:11:05.748262 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:05.748269 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l66d9\" (UniqueName: \"kubernetes.io/projected/9a162f09-5850-433e-bb8f-7e05f72c268d-kube-api-access-l66d9\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:11:06.291684 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:06.291647 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8" event={"ID":"9a162f09-5850-433e-bb8f-7e05f72c268d","Type":"ContainerDied","Data":"7ea6688d4e0279f5ca532606eeb658c3141f9e555230aa699e0d1c0e9cc7b3eb"} Jul 13 17:11:06.291872 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:06.291698 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7ea6688d4e0279f5ca532606eeb658c3141f9e555230aa699e0d1c0e9cc7b3eb" Jul 13 17:11:06.291872 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:06.291660 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbltg8" Jul 13 17:11:06.293280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:06.293246 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-kqszj" event={"ID":"2172fe62-0b80-4f9c-8820-dd3497b23640","Type":"ContainerStarted","Data":"779665d9d813a3c5b6d8499e7a7b0621ea6311ca26874545f0e986dc9ebabcd3"} Jul 13 17:11:06.293431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:06.293295 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-kqszj" Jul 13 17:11:06.294833 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:06.294804 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j5wdn" event={"ID":"a52fbba2-b14a-4d86-87ae-f45db2770d7e","Type":"ContainerStarted","Data":"9d8dbac934121137e6719f663bf5e18083f3c67c63f556ab2878df6ee5635574"} Jul 13 17:11:06.319590 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:06.319543 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-kqszj" podStartSLOduration=1.467114646 podStartE2EDuration="4.319526837s" totalImagesPullingTime="2.852412191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:11:06.318275304 +0000 UTC m=+343.981050492" watchObservedRunningTime="2026-07-13 17:11:06.319526837 +0000 UTC m=+343.982302024" Jul 13 17:11:06.346233 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:06.346178 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j5wdn" podStartSLOduration=1.470242603 podStartE2EDuration="4.346159637s" totalImagesPullingTime="2.875917034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:11:06.344081511 +0000 UTC m=+344.006856699" watchObservedRunningTime="2026-07-13 17:11:06.346159637 +0000 UTC m=+344.008934824" Jul 13 17:11:06.359465 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:06.359433 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:06.449528 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:06.449490 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:09.652975 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:09.652924 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:10.145023 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:10.144924 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:10.246253 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:10.246222 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:12.301313 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:12.301280 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-kqszj" Jul 13 17:11:17.175439 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:17.175358 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:17.274418 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:17.274375 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:19.533844 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:19.533813 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:19.870594 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:19.870509 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn"] Jul 13 17:11:19.870805 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:19.870792 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a162f09-5850-433e-bb8f-7e05f72c268d" containerName="extract" Jul 13 17:11:19.870855 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:19.870806 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a162f09-5850-433e-bb8f-7e05f72c268d" containerName="extract" Jul 13 17:11:19.870855 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:19.870818 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a162f09-5850-433e-bb8f-7e05f72c268d" containerName="pull" Jul 13 17:11:19.870855 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:19.870823 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a162f09-5850-433e-bb8f-7e05f72c268d" containerName="pull" Jul 13 17:11:19.870855 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:19.870839 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a162f09-5850-433e-bb8f-7e05f72c268d" containerName="util" Jul 13 17:11:19.870855 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:19.870844 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a162f09-5850-433e-bb8f-7e05f72c268d" containerName="util" Jul 13 17:11:19.870999 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:19.870894 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a162f09-5850-433e-bb8f-7e05f72c268d" containerName="extract" Jul 13 17:11:19.914515 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:19.914475 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn"] Jul 13 17:11:19.914681 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:19.914596 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn" Jul 13 17:11:19.917928 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:19.917903 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 17:11:19.918051 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:19.917903 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5gvvb\"" Jul 13 17:11:19.918051 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:19.917907 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 17:11:19.963850 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:19.963808 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6f79cb9-c98d-4c61-9cf6-38c01707748a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn\" (UID: \"b6f79cb9-c98d-4c61-9cf6-38c01707748a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn" Jul 13 17:11:19.964037 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:19.963909 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2cqk\" (UniqueName: \"kubernetes.io/projected/b6f79cb9-c98d-4c61-9cf6-38c01707748a-kube-api-access-v2cqk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn\" (UID: \"b6f79cb9-c98d-4c61-9cf6-38c01707748a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn" Jul 13 17:11:19.964037 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:19.963948 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6f79cb9-c98d-4c61-9cf6-38c01707748a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn\" (UID: \"b6f79cb9-c98d-4c61-9cf6-38c01707748a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn" Jul 13 17:11:20.064356 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:20.064311 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6f79cb9-c98d-4c61-9cf6-38c01707748a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn\" (UID: \"b6f79cb9-c98d-4c61-9cf6-38c01707748a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn" Jul 13 17:11:20.064532 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:20.064393 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2cqk\" (UniqueName: \"kubernetes.io/projected/b6f79cb9-c98d-4c61-9cf6-38c01707748a-kube-api-access-v2cqk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn\" (UID: \"b6f79cb9-c98d-4c61-9cf6-38c01707748a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn" Jul 13 17:11:20.064532 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:20.064420 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6f79cb9-c98d-4c61-9cf6-38c01707748a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn\" (UID: \"b6f79cb9-c98d-4c61-9cf6-38c01707748a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn" Jul 13 17:11:20.064819 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:20.064797 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6f79cb9-c98d-4c61-9cf6-38c01707748a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn\" (UID: \"b6f79cb9-c98d-4c61-9cf6-38c01707748a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn" Jul 13 17:11:20.064864 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:20.064807 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6f79cb9-c98d-4c61-9cf6-38c01707748a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn\" (UID: \"b6f79cb9-c98d-4c61-9cf6-38c01707748a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn" Jul 13 17:11:20.074198 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:20.074156 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2cqk\" (UniqueName: \"kubernetes.io/projected/b6f79cb9-c98d-4c61-9cf6-38c01707748a-kube-api-access-v2cqk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn\" (UID: \"b6f79cb9-c98d-4c61-9cf6-38c01707748a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn" Jul 13 17:11:20.223710 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:20.223668 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn" Jul 13 17:11:20.376236 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:20.376212 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn"] Jul 13 17:11:20.378483 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:11:20.378446 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6f79cb9_c98d_4c61_9cf6_38c01707748a.slice/crio-70d532ea0071be552d75a40b7320069d2293f4d24013fe6cf685026006327f20 WatchSource:0}: Error finding container 70d532ea0071be552d75a40b7320069d2293f4d24013fe6cf685026006327f20: Status 404 returned error can't find the container with id 70d532ea0071be552d75a40b7320069d2293f4d24013fe6cf685026006327f20 Jul 13 17:11:21.358584 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:21.358491 2564 generic.go:356] "Generic (PLEG): container finished" podID="b6f79cb9-c98d-4c61-9cf6-38c01707748a" containerID="43981b7fe008d8eb0e6e88686e08654a8497a2ccd274b02c18b1c0f21fccff6e" exitCode=0 Jul 13 17:11:21.358584 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:21.358577 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn" event={"ID":"b6f79cb9-c98d-4c61-9cf6-38c01707748a","Type":"ContainerDied","Data":"43981b7fe008d8eb0e6e88686e08654a8497a2ccd274b02c18b1c0f21fccff6e"} Jul 13 17:11:21.359017 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:21.358619 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn" event={"ID":"b6f79cb9-c98d-4c61-9cf6-38c01707748a","Type":"ContainerStarted","Data":"70d532ea0071be552d75a40b7320069d2293f4d24013fe6cf685026006327f20"} Jul 13 17:11:22.365233 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:22.365144 2564 generic.go:356] "Generic (PLEG): container finished" podID="b6f79cb9-c98d-4c61-9cf6-38c01707748a" containerID="2919de64b76a2780604e4425f86df4c1b2daff99b9c90fd39f68caebc40eb934" exitCode=0 Jul 13 17:11:22.365590 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:22.365232 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn" event={"ID":"b6f79cb9-c98d-4c61-9cf6-38c01707748a","Type":"ContainerDied","Data":"2919de64b76a2780604e4425f86df4c1b2daff99b9c90fd39f68caebc40eb934"} Jul 13 17:11:23.371300 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:23.371258 2564 generic.go:356] "Generic (PLEG): container finished" podID="b6f79cb9-c98d-4c61-9cf6-38c01707748a" containerID="96e12423d0e7620330dfdce56a962f7819b689b4098f6a2890c9c2272051fd56" exitCode=0 Jul 13 17:11:23.371691 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:23.371314 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn" event={"ID":"b6f79cb9-c98d-4c61-9cf6-38c01707748a","Type":"ContainerDied","Data":"96e12423d0e7620330dfdce56a962f7819b689b4098f6a2890c9c2272051fd56"} Jul 13 17:11:24.505012 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:24.504987 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn" Jul 13 17:11:24.601007 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:24.600966 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6f79cb9-c98d-4c61-9cf6-38c01707748a-util\") pod \"b6f79cb9-c98d-4c61-9cf6-38c01707748a\" (UID: \"b6f79cb9-c98d-4c61-9cf6-38c01707748a\") " Jul 13 17:11:24.601204 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:24.601020 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6f79cb9-c98d-4c61-9cf6-38c01707748a-bundle\") pod \"b6f79cb9-c98d-4c61-9cf6-38c01707748a\" (UID: \"b6f79cb9-c98d-4c61-9cf6-38c01707748a\") " Jul 13 17:11:24.601204 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:24.601074 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2cqk\" (UniqueName: \"kubernetes.io/projected/b6f79cb9-c98d-4c61-9cf6-38c01707748a-kube-api-access-v2cqk\") pod \"b6f79cb9-c98d-4c61-9cf6-38c01707748a\" (UID: \"b6f79cb9-c98d-4c61-9cf6-38c01707748a\") " Jul 13 17:11:24.601851 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:24.601820 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f79cb9-c98d-4c61-9cf6-38c01707748a-bundle" (OuterVolumeSpecName: "bundle") pod "b6f79cb9-c98d-4c61-9cf6-38c01707748a" (UID: "b6f79cb9-c98d-4c61-9cf6-38c01707748a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:11:24.603341 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:24.603318 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6f79cb9-c98d-4c61-9cf6-38c01707748a-kube-api-access-v2cqk" (OuterVolumeSpecName: "kube-api-access-v2cqk") pod "b6f79cb9-c98d-4c61-9cf6-38c01707748a" (UID: "b6f79cb9-c98d-4c61-9cf6-38c01707748a"). InnerVolumeSpecName "kube-api-access-v2cqk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:11:24.607033 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:24.606936 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f79cb9-c98d-4c61-9cf6-38c01707748a-util" (OuterVolumeSpecName: "util") pod "b6f79cb9-c98d-4c61-9cf6-38c01707748a" (UID: "b6f79cb9-c98d-4c61-9cf6-38c01707748a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:11:24.702370 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:24.702339 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6f79cb9-c98d-4c61-9cf6-38c01707748a-util\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:11:24.702370 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:24.702366 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6f79cb9-c98d-4c61-9cf6-38c01707748a-bundle\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:11:24.702555 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:24.702377 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2cqk\" (UniqueName: \"kubernetes.io/projected/b6f79cb9-c98d-4c61-9cf6-38c01707748a-kube-api-access-v2cqk\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:11:25.381390 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:25.381289 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn" event={"ID":"b6f79cb9-c98d-4c61-9cf6-38c01707748a","Type":"ContainerDied","Data":"70d532ea0071be552d75a40b7320069d2293f4d24013fe6cf685026006327f20"} Jul 13 17:11:25.381390 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:25.381320 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af22ktn" Jul 13 17:11:25.381390 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:25.381338 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70d532ea0071be552d75a40b7320069d2293f4d24013fe6cf685026006327f20" Jul 13 17:11:30.982267 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:30.982229 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:31.069947 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.069912 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:31.564345 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.564307 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:31.785058 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.785026 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:31.918921 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.918888 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:31.947305 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.947269 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54"] Jul 13 17:11:31.947544 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.947531 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f79cb9-c98d-4c61-9cf6-38c01707748a" containerName="util" Jul 13 17:11:31.947588 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.947546 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f79cb9-c98d-4c61-9cf6-38c01707748a" containerName="util" Jul 13 17:11:31.947588 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.947556 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f79cb9-c98d-4c61-9cf6-38c01707748a" containerName="pull" Jul 13 17:11:31.947588 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.947562 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f79cb9-c98d-4c61-9cf6-38c01707748a" containerName="pull" Jul 13 17:11:31.947588 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.947577 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f79cb9-c98d-4c61-9cf6-38c01707748a" containerName="extract" Jul 13 17:11:31.947588 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.947582 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f79cb9-c98d-4c61-9cf6-38c01707748a" containerName="extract" Jul 13 17:11:31.947744 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.947623 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6f79cb9-c98d-4c61-9cf6-38c01707748a" containerName="extract" Jul 13 17:11:31.949953 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.949936 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:31.953260 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.953234 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 13 17:11:31.953413 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.953286 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 13 17:11:31.953413 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.953301 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 13 17:11:31.953413 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.953310 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 13 17:11:31.953681 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.953665 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kx26v\"" Jul 13 17:11:31.972167 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:31.972136 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54"] Jul 13 17:11:32.059753 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.059688 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c09bd7aa-9223-4fed-bd75-2578ec1605b3-webhook-cert\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-lzk54\" (UID: \"c09bd7aa-9223-4fed-bd75-2578ec1605b3\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:32.060183 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.059880 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2v8b\" (UniqueName: \"kubernetes.io/projected/c09bd7aa-9223-4fed-bd75-2578ec1605b3-kube-api-access-k2v8b\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-lzk54\" (UID: \"c09bd7aa-9223-4fed-bd75-2578ec1605b3\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:32.060183 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.059927 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c09bd7aa-9223-4fed-bd75-2578ec1605b3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-lzk54\" (UID: \"c09bd7aa-9223-4fed-bd75-2578ec1605b3\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:32.060183 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.059968 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c09bd7aa-9223-4fed-bd75-2578ec1605b3-tmp\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-lzk54\" (UID: \"c09bd7aa-9223-4fed-bd75-2578ec1605b3\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:32.060183 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.060003 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c09bd7aa-9223-4fed-bd75-2578ec1605b3-manifests\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-lzk54\" (UID: \"c09bd7aa-9223-4fed-bd75-2578ec1605b3\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:32.118974 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.118929 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:32.161234 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.161197 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c09bd7aa-9223-4fed-bd75-2578ec1605b3-webhook-cert\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-lzk54\" (UID: \"c09bd7aa-9223-4fed-bd75-2578ec1605b3\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:32.161394 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.161252 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2v8b\" (UniqueName: \"kubernetes.io/projected/c09bd7aa-9223-4fed-bd75-2578ec1605b3-kube-api-access-k2v8b\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-lzk54\" (UID: \"c09bd7aa-9223-4fed-bd75-2578ec1605b3\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:32.161394 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.161281 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c09bd7aa-9223-4fed-bd75-2578ec1605b3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-lzk54\" (UID: \"c09bd7aa-9223-4fed-bd75-2578ec1605b3\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:32.161394 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.161313 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c09bd7aa-9223-4fed-bd75-2578ec1605b3-tmp\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-lzk54\" (UID: \"c09bd7aa-9223-4fed-bd75-2578ec1605b3\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:32.161394 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.161342 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c09bd7aa-9223-4fed-bd75-2578ec1605b3-manifests\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-lzk54\" (UID: \"c09bd7aa-9223-4fed-bd75-2578ec1605b3\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:32.161799 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.161764 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c09bd7aa-9223-4fed-bd75-2578ec1605b3-manifests\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-lzk54\" (UID: \"c09bd7aa-9223-4fed-bd75-2578ec1605b3\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:32.163780 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.163751 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c09bd7aa-9223-4fed-bd75-2578ec1605b3-tmp\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-lzk54\" (UID: \"c09bd7aa-9223-4fed-bd75-2578ec1605b3\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:32.163973 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.163953 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c09bd7aa-9223-4fed-bd75-2578ec1605b3-webhook-cert\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-lzk54\" (UID: \"c09bd7aa-9223-4fed-bd75-2578ec1605b3\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:32.164017 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.163966 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c09bd7aa-9223-4fed-bd75-2578ec1605b3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-lzk54\" (UID: \"c09bd7aa-9223-4fed-bd75-2578ec1605b3\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:32.171136 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.171083 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2v8b\" (UniqueName: \"kubernetes.io/projected/c09bd7aa-9223-4fed-bd75-2578ec1605b3-kube-api-access-k2v8b\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-lzk54\" (UID: \"c09bd7aa-9223-4fed-bd75-2578ec1605b3\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:32.220040 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.220008 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:32.260343 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.260312 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:32.321581 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.321093 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:32.415158 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.415135 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54"] Jul 13 17:11:32.417034 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:11:32.417007 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc09bd7aa_9223_4fed_bd75_2578ec1605b3.slice/crio-b3dddf5b39021d8c0b8740434cd6a2fb5acabc31bec8288867bfc125de1e32c7 WatchSource:0}: Error finding container b3dddf5b39021d8c0b8740434cd6a2fb5acabc31bec8288867bfc125de1e32c7: Status 404 returned error can't find the container with id b3dddf5b39021d8c0b8740434cd6a2fb5acabc31bec8288867bfc125de1e32c7 Jul 13 17:11:32.427220 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.427195 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:32.493633 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:32.493602 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:33.416656 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:33.416608 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" event={"ID":"c09bd7aa-9223-4fed-bd75-2578ec1605b3","Type":"ContainerStarted","Data":"b3dddf5b39021d8c0b8740434cd6a2fb5acabc31bec8288867bfc125de1e32c7"} Jul 13 17:11:36.432867 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:36.432825 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" event={"ID":"c09bd7aa-9223-4fed-bd75-2578ec1605b3","Type":"ContainerStarted","Data":"e8c273216d5934a030a004d8358f13df7f1f3e10ddb258e39e6cefbbd562b538"} Jul 13 17:11:37.438118 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:37.438078 2564 generic.go:356] "Generic (PLEG): container finished" podID="c09bd7aa-9223-4fed-bd75-2578ec1605b3" containerID="e8c273216d5934a030a004d8358f13df7f1f3e10ddb258e39e6cefbbd562b538" exitCode=0 Jul 13 17:11:37.438118 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:37.438121 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" event={"ID":"c09bd7aa-9223-4fed-bd75-2578ec1605b3","Type":"ContainerDied","Data":"e8c273216d5934a030a004d8358f13df7f1f3e10ddb258e39e6cefbbd562b538"} Jul 13 17:11:38.444200 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:38.444166 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" event={"ID":"c09bd7aa-9223-4fed-bd75-2578ec1605b3","Type":"ContainerStarted","Data":"97f43ad45f871a02e9f69d7a2e425c520e168d0b8a5102d1474d88f761112799"} Jul 13 17:11:38.444578 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:38.444285 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:38.475055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:38.475002 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" podStartSLOduration=2.675778208 podStartE2EDuration="7.4749806s" totalImagesPullingTime="3.793189211s" totalInitContainerRuntime="1.006013181s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:11:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:11:38.473788195 +0000 UTC m=+376.136563376" watchObservedRunningTime="2026-07-13 17:11:38.4749806 +0000 UTC m=+376.137755788" Jul 13 17:11:45.771061 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:45.771019 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:45.892048 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:45.892017 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:49.451072 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:49.451038 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-lzk54" Jul 13 17:11:51.594309 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:51.594273 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:51.793709 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:51.793677 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:52.102928 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.102899 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:52.203147 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.203114 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-746dfd5849-cm44k"] Jul 13 17:11:52.210107 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.210077 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-746dfd5849-cm44k" Jul 13 17:11:52.215184 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.215159 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 13 17:11:52.216857 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.216829 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-wpp9f\"" Jul 13 17:11:52.217001 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.216833 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 13 17:11:52.217001 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.216913 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 13 17:11:52.217001 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.216948 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 13 17:11:52.227381 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.227354 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-746dfd5849-cm44k"] Jul 13 17:11:52.326889 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.326858 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njkf7\" (UniqueName: \"kubernetes.io/projected/098a1fa8-7634-46cf-a499-5d6b3c627fb6-kube-api-access-njkf7\") pod \"kube-auth-proxy-746dfd5849-cm44k\" (UID: \"098a1fa8-7634-46cf-a499-5d6b3c627fb6\") " pod="openshift-ingress/kube-auth-proxy-746dfd5849-cm44k" Jul 13 17:11:52.327054 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.326955 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/098a1fa8-7634-46cf-a499-5d6b3c627fb6-tls-certs\") pod \"kube-auth-proxy-746dfd5849-cm44k\" (UID: \"098a1fa8-7634-46cf-a499-5d6b3c627fb6\") " pod="openshift-ingress/kube-auth-proxy-746dfd5849-cm44k" Jul 13 17:11:52.327054 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.326999 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/098a1fa8-7634-46cf-a499-5d6b3c627fb6-tmp\") pod \"kube-auth-proxy-746dfd5849-cm44k\" (UID: \"098a1fa8-7634-46cf-a499-5d6b3c627fb6\") " pod="openshift-ingress/kube-auth-proxy-746dfd5849-cm44k" Jul 13 17:11:52.391951 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.391863 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:52.419160 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.419123 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-9j2b6"] Jul 13 17:11:52.421478 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.421461 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-9j2b6" Jul 13 17:11:52.424343 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.424320 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 13 17:11:52.424969 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.424942 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-dmpzv\"" Jul 13 17:11:52.427458 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.427432 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/098a1fa8-7634-46cf-a499-5d6b3c627fb6-tls-certs\") pod \"kube-auth-proxy-746dfd5849-cm44k\" (UID: \"098a1fa8-7634-46cf-a499-5d6b3c627fb6\") " pod="openshift-ingress/kube-auth-proxy-746dfd5849-cm44k" Jul 13 17:11:52.427589 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.427487 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/098a1fa8-7634-46cf-a499-5d6b3c627fb6-tmp\") pod \"kube-auth-proxy-746dfd5849-cm44k\" (UID: \"098a1fa8-7634-46cf-a499-5d6b3c627fb6\") " pod="openshift-ingress/kube-auth-proxy-746dfd5849-cm44k" Jul 13 17:11:52.427589 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.427555 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njkf7\" (UniqueName: \"kubernetes.io/projected/098a1fa8-7634-46cf-a499-5d6b3c627fb6-kube-api-access-njkf7\") pod \"kube-auth-proxy-746dfd5849-cm44k\" (UID: \"098a1fa8-7634-46cf-a499-5d6b3c627fb6\") " pod="openshift-ingress/kube-auth-proxy-746dfd5849-cm44k" Jul 13 17:11:52.430263 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.430242 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/098a1fa8-7634-46cf-a499-5d6b3c627fb6-tmp\") pod \"kube-auth-proxy-746dfd5849-cm44k\" (UID: \"098a1fa8-7634-46cf-a499-5d6b3c627fb6\") " pod="openshift-ingress/kube-auth-proxy-746dfd5849-cm44k" Jul 13 17:11:52.430393 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.430373 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/098a1fa8-7634-46cf-a499-5d6b3c627fb6-tls-certs\") pod \"kube-auth-proxy-746dfd5849-cm44k\" (UID: \"098a1fa8-7634-46cf-a499-5d6b3c627fb6\") " pod="openshift-ingress/kube-auth-proxy-746dfd5849-cm44k" Jul 13 17:11:52.440098 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.440069 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-9j2b6"] Jul 13 17:11:52.445823 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.445796 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njkf7\" (UniqueName: \"kubernetes.io/projected/098a1fa8-7634-46cf-a499-5d6b3c627fb6-kube-api-access-njkf7\") pod \"kube-auth-proxy-746dfd5849-cm44k\" (UID: \"098a1fa8-7634-46cf-a499-5d6b3c627fb6\") " pod="openshift-ingress/kube-auth-proxy-746dfd5849-cm44k" Jul 13 17:11:52.486276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.486232 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:52.519806 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.519751 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-746dfd5849-cm44k" Jul 13 17:11:52.528310 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.528280 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3aec7a8a-c182-4055-8818-e23a9e1645d0-cert\") pod \"odh-model-controller-7998c67dc4-9j2b6\" (UID: \"3aec7a8a-c182-4055-8818-e23a9e1645d0\") " pod="opendatahub/odh-model-controller-7998c67dc4-9j2b6" Jul 13 17:11:52.528467 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.528334 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4lrn\" (UniqueName: \"kubernetes.io/projected/3aec7a8a-c182-4055-8818-e23a9e1645d0-kube-api-access-f4lrn\") pod \"odh-model-controller-7998c67dc4-9j2b6\" (UID: \"3aec7a8a-c182-4055-8818-e23a9e1645d0\") " pod="opendatahub/odh-model-controller-7998c67dc4-9j2b6" Jul 13 17:11:52.612376 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.612327 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:52.628910 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.628842 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4lrn\" (UniqueName: \"kubernetes.io/projected/3aec7a8a-c182-4055-8818-e23a9e1645d0-kube-api-access-f4lrn\") pod \"odh-model-controller-7998c67dc4-9j2b6\" (UID: \"3aec7a8a-c182-4055-8818-e23a9e1645d0\") " pod="opendatahub/odh-model-controller-7998c67dc4-9j2b6" Jul 13 17:11:52.629120 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.628985 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3aec7a8a-c182-4055-8818-e23a9e1645d0-cert\") pod \"odh-model-controller-7998c67dc4-9j2b6\" (UID: \"3aec7a8a-c182-4055-8818-e23a9e1645d0\") " pod="opendatahub/odh-model-controller-7998c67dc4-9j2b6" Jul 13 17:11:52.629183 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:11:52.629127 2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 13 17:11:52.629226 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:11:52.629202 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3aec7a8a-c182-4055-8818-e23a9e1645d0-cert podName:3aec7a8a-c182-4055-8818-e23a9e1645d0 nodeName:}" failed. No retries permitted until 2026-07-13 17:11:53.129180797 +0000 UTC m=+390.791955967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3aec7a8a-c182-4055-8818-e23a9e1645d0-cert") pod "odh-model-controller-7998c67dc4-9j2b6" (UID: "3aec7a8a-c182-4055-8818-e23a9e1645d0") : secret "odh-model-controller-webhook-cert" not found Jul 13 17:11:52.638982 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.638953 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4lrn\" (UniqueName: \"kubernetes.io/projected/3aec7a8a-c182-4055-8818-e23a9e1645d0-kube-api-access-f4lrn\") pod \"odh-model-controller-7998c67dc4-9j2b6\" (UID: \"3aec7a8a-c182-4055-8818-e23a9e1645d0\") " pod="opendatahub/odh-model-controller-7998c67dc4-9j2b6" Jul 13 17:11:52.659343 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:52.659321 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-746dfd5849-cm44k"] Jul 13 17:11:52.661654 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:11:52.661628 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod098a1fa8_7634_46cf_a499_5d6b3c627fb6.slice/crio-9361fd87aee46b8a2f389d03428ea362493fa089c14b6763d32a5184fc08314e WatchSource:0}: Error finding container 9361fd87aee46b8a2f389d03428ea362493fa089c14b6763d32a5184fc08314e: Status 404 returned error can't find the container with id 9361fd87aee46b8a2f389d03428ea362493fa089c14b6763d32a5184fc08314e Jul 13 17:11:53.132704 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.132615 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3aec7a8a-c182-4055-8818-e23a9e1645d0-cert\") pod \"odh-model-controller-7998c67dc4-9j2b6\" (UID: \"3aec7a8a-c182-4055-8818-e23a9e1645d0\") " pod="opendatahub/odh-model-controller-7998c67dc4-9j2b6" Jul 13 17:11:53.132907 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:11:53.132841 2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 13 17:11:53.132993 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:11:53.132919 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3aec7a8a-c182-4055-8818-e23a9e1645d0-cert podName:3aec7a8a-c182-4055-8818-e23a9e1645d0 nodeName:}" failed. No retries permitted until 2026-07-13 17:11:54.132897664 +0000 UTC m=+391.795672843 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3aec7a8a-c182-4055-8818-e23a9e1645d0-cert") pod "odh-model-controller-7998c67dc4-9j2b6" (UID: "3aec7a8a-c182-4055-8818-e23a9e1645d0") : secret "odh-model-controller-webhook-cert" not found Jul 13 17:11:53.202327 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.202272 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:53.306455 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.306420 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:53.318895 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.318859 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7hstg"] Jul 13 17:11:53.322107 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.322081 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7hstg" Jul 13 17:11:53.327678 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.327649 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-svrrv\"" Jul 13 17:11:53.327897 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.327650 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 13 17:11:53.360978 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.360940 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7hstg"] Jul 13 17:11:53.435524 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.435489 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpkjk\" (UniqueName: \"kubernetes.io/projected/d7042803-711a-4efb-a0ac-5a1825bc5d7d-kube-api-access-fpkjk\") pod \"model-serving-api-54d89bd5fc-7hstg\" (UID: \"d7042803-711a-4efb-a0ac-5a1825bc5d7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-7hstg" Jul 13 17:11:53.435707 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.435539 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7042803-711a-4efb-a0ac-5a1825bc5d7d-tls-certs\") pod \"model-serving-api-54d89bd5fc-7hstg\" (UID: \"d7042803-711a-4efb-a0ac-5a1825bc5d7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-7hstg" Jul 13 17:11:53.511139 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.511100 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-746dfd5849-cm44k" event={"ID":"098a1fa8-7634-46cf-a499-5d6b3c627fb6","Type":"ContainerStarted","Data":"9361fd87aee46b8a2f389d03428ea362493fa089c14b6763d32a5184fc08314e"} Jul 13 17:11:53.521871 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.521833 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:53.537044 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.537003 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpkjk\" (UniqueName: \"kubernetes.io/projected/d7042803-711a-4efb-a0ac-5a1825bc5d7d-kube-api-access-fpkjk\") pod \"model-serving-api-54d89bd5fc-7hstg\" (UID: \"d7042803-711a-4efb-a0ac-5a1825bc5d7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-7hstg" Jul 13 17:11:53.537044 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.537055 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7042803-711a-4efb-a0ac-5a1825bc5d7d-tls-certs\") pod \"model-serving-api-54d89bd5fc-7hstg\" (UID: \"d7042803-711a-4efb-a0ac-5a1825bc5d7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-7hstg" Jul 13 17:11:53.539968 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.539938 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7042803-711a-4efb-a0ac-5a1825bc5d7d-tls-certs\") pod \"model-serving-api-54d89bd5fc-7hstg\" (UID: \"d7042803-711a-4efb-a0ac-5a1825bc5d7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-7hstg" Jul 13 17:11:53.548096 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.548066 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpkjk\" (UniqueName: \"kubernetes.io/projected/d7042803-711a-4efb-a0ac-5a1825bc5d7d-kube-api-access-fpkjk\") pod \"model-serving-api-54d89bd5fc-7hstg\" (UID: \"d7042803-711a-4efb-a0ac-5a1825bc5d7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-7hstg" Jul 13 17:11:53.636144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.636102 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7hstg" Jul 13 17:11:53.757668 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.757632 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:53.766626 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.766593 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh"] Jul 13 17:11:53.770068 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.770044 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.774297 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.774276 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 13 17:11:53.774422 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.774279 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 13 17:11:53.774673 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.774652 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 13 17:11:53.774879 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.774866 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-xbd2j\"" Jul 13 17:11:53.774960 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.774889 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 13 17:11:53.791147 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.791108 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7hstg"] Jul 13 17:11:53.791646 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:11:53.791615 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7042803_711a_4efb_a0ac_5a1825bc5d7d.slice/crio-7d3952afe825896e03615236bd05a8aee81aa84c6ab31c4961057bbe8c959233 WatchSource:0}: Error finding container 7d3952afe825896e03615236bd05a8aee81aa84c6ab31c4961057bbe8c959233: Status 404 returned error can't find the container with id 7d3952afe825896e03615236bd05a8aee81aa84c6ab31c4961057bbe8c959233 Jul 13 17:11:53.793753 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.793692 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh"] Jul 13 17:11:53.840652 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.840614 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.840856 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.840683 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-token\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.840856 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.840710 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c05b61e5-b79e-4b55-8d79-3f757abd0384-cacerts\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.840856 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.840782 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.840856 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.840827 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm669\" (UniqueName: \"kubernetes.io/projected/c05b61e5-b79e-4b55-8d79-3f757abd0384-kube-api-access-hm669\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.841011 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.840861 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.841011 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.840889 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c05b61e5-b79e-4b55-8d79-3f757abd0384-local-certs\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.858892 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.858860 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:53.941344 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.941306 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm669\" (UniqueName: \"kubernetes.io/projected/c05b61e5-b79e-4b55-8d79-3f757abd0384-kube-api-access-hm669\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.941344 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.941344 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.941550 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.941372 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c05b61e5-b79e-4b55-8d79-3f757abd0384-local-certs\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.941550 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.941414 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.941644 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.941588 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-token\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.941696 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.941662 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c05b61e5-b79e-4b55-8d79-3f757abd0384-cacerts\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.941696 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.941687 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.942020 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.941994 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.944022 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.943955 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.944022 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.944003 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.944322 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.944305 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c05b61e5-b79e-4b55-8d79-3f757abd0384-local-certs\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.944434 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.944419 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c05b61e5-b79e-4b55-8d79-3f757abd0384-cacerts\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.953306 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.953278 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-token\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:53.953432 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:53.953414 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm669\" (UniqueName: \"kubernetes.io/projected/c05b61e5-b79e-4b55-8d79-3f757abd0384-kube-api-access-hm669\") pod \"istiod-openshift-gateway-7574459656-cmxsh\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:54.080150 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:54.080109 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:54.144064 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:54.143985 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3aec7a8a-c182-4055-8818-e23a9e1645d0-cert\") pod \"odh-model-controller-7998c67dc4-9j2b6\" (UID: \"3aec7a8a-c182-4055-8818-e23a9e1645d0\") " pod="opendatahub/odh-model-controller-7998c67dc4-9j2b6" Jul 13 17:11:54.146507 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:54.146480 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3aec7a8a-c182-4055-8818-e23a9e1645d0-cert\") pod \"odh-model-controller-7998c67dc4-9j2b6\" (UID: \"3aec7a8a-c182-4055-8818-e23a9e1645d0\") " pod="opendatahub/odh-model-controller-7998c67dc4-9j2b6" Jul 13 17:11:54.226034 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:54.225940 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh"] Jul 13 17:11:54.229702 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:11:54.229636 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc05b61e5_b79e_4b55_8d79_3f757abd0384.slice/crio-6b55a84578eb400c139fbd7794b149e534befb0425de918573c8537ed16aec2e WatchSource:0}: Error finding container 6b55a84578eb400c139fbd7794b149e534befb0425de918573c8537ed16aec2e: Status 404 returned error can't find the container with id 6b55a84578eb400c139fbd7794b149e534befb0425de918573c8537ed16aec2e Jul 13 17:11:54.232373 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:54.232115 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-9j2b6" Jul 13 17:11:54.391950 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:54.391921 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-9j2b6"] Jul 13 17:11:54.393837 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:11:54.393801 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aec7a8a_c182_4055_8818_e23a9e1645d0.slice/crio-d1bb62813a9ae8296f6ac1c02916b5cdb368b3709d6891de4dbb934cb06c876a WatchSource:0}: Error finding container d1bb62813a9ae8296f6ac1c02916b5cdb368b3709d6891de4dbb934cb06c876a: Status 404 returned error can't find the container with id d1bb62813a9ae8296f6ac1c02916b5cdb368b3709d6891de4dbb934cb06c876a Jul 13 17:11:54.518549 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:54.518449 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-9j2b6" event={"ID":"3aec7a8a-c182-4055-8818-e23a9e1645d0","Type":"ContainerStarted","Data":"d1bb62813a9ae8296f6ac1c02916b5cdb368b3709d6891de4dbb934cb06c876a"} Jul 13 17:11:54.519975 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:54.519947 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7hstg" event={"ID":"d7042803-711a-4efb-a0ac-5a1825bc5d7d","Type":"ContainerStarted","Data":"7d3952afe825896e03615236bd05a8aee81aa84c6ab31c4961057bbe8c959233"} Jul 13 17:11:54.521372 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:54.521345 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" event={"ID":"c05b61e5-b79e-4b55-8d79-3f757abd0384","Type":"ContainerStarted","Data":"6b55a84578eb400c139fbd7794b149e534befb0425de918573c8537ed16aec2e"} Jul 13 17:11:54.942833 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:54.942794 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:11:56.534747 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:56.534691 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-746dfd5849-cm44k" event={"ID":"098a1fa8-7634-46cf-a499-5d6b3c627fb6","Type":"ContainerStarted","Data":"f7bd198df6b24bcee9e6d4818f8c2df53be3f8fb3657a65f5fdf9a0fba7b6169"} Jul 13 17:11:56.552889 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:56.552455 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-746dfd5849-cm44k" podStartSLOduration=1.6019755180000002 podStartE2EDuration="4.552430876s" totalImagesPullingTime="2.950455358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:11:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:11:56.551509882 +0000 UTC m=+394.214285101" watchObservedRunningTime="2026-07-13 17:11:56.552430876 +0000 UTC m=+394.215206068" Jul 13 17:11:57.543677 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:57.543610 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7hstg" event={"ID":"d7042803-711a-4efb-a0ac-5a1825bc5d7d","Type":"ContainerStarted","Data":"49617c50113850aab20083791b9cbaf092d37cd75f82a82842d57e0f473b9320"} Jul 13 17:11:57.544151 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:57.544020 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-7hstg" Jul 13 17:11:57.564422 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:57.564356 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-7hstg" podStartSLOduration=1.4406105710000001 podStartE2EDuration="4.56433417s" totalImagesPullingTime="3.123723599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:11:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:11:57.562216203 +0000 UTC m=+395.224991393" watchObservedRunningTime="2026-07-13 17:11:57.56433417 +0000 UTC m=+395.227109359" Jul 13 17:11:59.300711 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:59.300649 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:11:59.301138 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:59.300768 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:11:59.554813 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:59.554694 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" event={"ID":"c05b61e5-b79e-4b55-8d79-3f757abd0384","Type":"ContainerStarted","Data":"362d8bccd019d655a3da79139fe8b92bc23f4f7bfbbf22da7646fcdd3cb361ea"} Jul 13 17:11:59.554989 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:59.554824 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:11:59.556543 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:59.556512 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-9j2b6" event={"ID":"3aec7a8a-c182-4055-8818-e23a9e1645d0","Type":"ContainerStarted","Data":"48248ac53431082043b9cdff722e067ae3cac350d9a72fac960b8fbf3950e0ba"} Jul 13 17:11:59.556664 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:59.556599 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-9j2b6" Jul 13 17:11:59.580764 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:59.580699 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" podStartSLOduration=1.5125700100000001 podStartE2EDuration="6.580681605s" totalImagesPullingTime="5.068111595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:11:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:11:59.578130748 +0000 UTC m=+397.240905938" watchObservedRunningTime="2026-07-13 17:11:59.580681605 +0000 UTC m=+397.243456793" Jul 13 17:11:59.605567 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:11:59.605508 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-9j2b6" podStartSLOduration=2.671420243 podStartE2EDuration="7.605491243s" totalImagesPullingTime="4.934071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:11:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:11:59.604323614 +0000 UTC m=+397.267098801" watchObservedRunningTime="2026-07-13 17:11:59.605491243 +0000 UTC m=+397.268266433" Jul 13 17:12:00.037522 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:00.037484 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:12:00.562641 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:00.562606 2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-cmxsh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 17:12:00.563035 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:00.562676 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" podUID="c05b61e5-b79e-4b55-8d79-3f757abd0384" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 17:12:00.926123 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:00.926085 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx"] Jul 13 17:12:00.928625 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:00.928608 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:00.932514 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:00.932491 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-9l7cd\"" Jul 13 17:12:00.953519 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:00.953481 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx"] Jul 13 17:12:01.012890 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.012848 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.013098 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.012914 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.013098 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.012963 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.013098 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.013026 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.013098 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.013049 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.013098 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.013073 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.013288 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.013119 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrk7b\" (UniqueName: \"kubernetes.io/projected/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-kube-api-access-mrk7b\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.013288 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.013136 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.013288 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.013182 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.025613 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.025575 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:12:01.114113 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.114075 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.114113 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.114120 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.114345 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.114188 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.114345 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.114232 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.114345 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.114250 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.114345 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.114307 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrk7b\" (UniqueName: \"kubernetes.io/projected/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-kube-api-access-mrk7b\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.114345 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.114336 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.114542 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.114360 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.114542 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.114448 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.114654 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.114621 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.114713 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.114659 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.114979 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.114958 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.115069 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.115048 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.115791 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.115768 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.116891 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.116869 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.116891 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.116880 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.124038 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.124013 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.124168 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.124142 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrk7b\" (UniqueName: \"kubernetes.io/projected/c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13-kube-api-access-mrk7b\") pod \"data-science-gateway-data-science-gateway-class-6576859cf449pjx\" (UID: \"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.205143 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.205068 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:12:01.238236 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.238199 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:01.249550 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.249524 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz"] Jul 13 17:12:01.252178 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.252152 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.278466 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.278432 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz"] Jul 13 17:12:01.315677 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.315645 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a269ed30-312c-421b-aa7e-d45f32aab007-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.315846 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.315696 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a269ed30-312c-421b-aa7e-d45f32aab007-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.315846 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.315823 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a269ed30-312c-421b-aa7e-d45f32aab007-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.315936 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.315857 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a269ed30-312c-421b-aa7e-d45f32aab007-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.315936 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.315888 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a269ed30-312c-421b-aa7e-d45f32aab007-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.316004 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.315979 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4vh9\" (UniqueName: \"kubernetes.io/projected/a269ed30-312c-421b-aa7e-d45f32aab007-kube-api-access-g4vh9\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.316041 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.316032 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a269ed30-312c-421b-aa7e-d45f32aab007-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.390647 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.390617 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx"] Jul 13 17:12:01.392703 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:12:01.392662 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc52307b7_2e8c_46b0_90bb_8ff1ba9e0c13.slice/crio-5683c24b3452a37b72561fe6ed7888e8c26180e5c4d7ee80d679a0756643fa6a WatchSource:0}: Error finding container 5683c24b3452a37b72561fe6ed7888e8c26180e5c4d7ee80d679a0756643fa6a: Status 404 returned error can't find the container with id 5683c24b3452a37b72561fe6ed7888e8c26180e5c4d7ee80d679a0756643fa6a Jul 13 17:12:01.417452 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.417415 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a269ed30-312c-421b-aa7e-d45f32aab007-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.417595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.417458 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a269ed30-312c-421b-aa7e-d45f32aab007-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.417648 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.417594 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a269ed30-312c-421b-aa7e-d45f32aab007-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.417709 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.417670 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4vh9\" (UniqueName: \"kubernetes.io/projected/a269ed30-312c-421b-aa7e-d45f32aab007-kube-api-access-g4vh9\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.417786 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.417752 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a269ed30-312c-421b-aa7e-d45f32aab007-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.417841 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.417793 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a269ed30-312c-421b-aa7e-d45f32aab007-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.417841 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.417831 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a269ed30-312c-421b-aa7e-d45f32aab007-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.418855 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.418786 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a269ed30-312c-421b-aa7e-d45f32aab007-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.420800 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.420634 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a269ed30-312c-421b-aa7e-d45f32aab007-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.420800 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.420708 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a269ed30-312c-421b-aa7e-d45f32aab007-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.420984 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.420835 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a269ed30-312c-421b-aa7e-d45f32aab007-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.420984 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.420942 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a269ed30-312c-421b-aa7e-d45f32aab007-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.427022 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.427000 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a269ed30-312c-421b-aa7e-d45f32aab007-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.428163 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.428144 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4vh9\" (UniqueName: \"kubernetes.io/projected/a269ed30-312c-421b-aa7e-d45f32aab007-kube-api-access-g4vh9\") pod \"istiod-openshift-gateway-94bb8fbfd-cf8tz\" (UID: \"a269ed30-312c-421b-aa7e-d45f32aab007\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.567039 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.566948 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" event={"ID":"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13","Type":"ContainerStarted","Data":"5683c24b3452a37b72561fe6ed7888e8c26180e5c4d7ee80d679a0756643fa6a"} Jul 13 17:12:01.569282 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.569258 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:01.719445 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.719406 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz"] Jul 13 17:12:01.722261 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.722230 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:12:01.722621 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:12:01.722598 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda269ed30_312c_421b_aa7e_d45f32aab007.slice/crio-c66b18313780900dd64e3b18e4e2ab63525b393d8fc2abbbc513e3e289cb5a5d WatchSource:0}: Error finding container c66b18313780900dd64e3b18e4e2ab63525b393d8fc2abbbc513e3e289cb5a5d: Status 404 returned error can't find the container with id c66b18313780900dd64e3b18e4e2ab63525b393d8fc2abbbc513e3e289cb5a5d Jul 13 17:12:01.726858 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.726805 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:12:01.726951 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:01.726887 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:12:02.580804 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:02.580763 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" event={"ID":"a269ed30-312c-421b-aa7e-d45f32aab007","Type":"ContainerStarted","Data":"05ee6d633f4445af31d8fbd98efc0e9a99e0fbf3b8ee383b059eb3fcc0e3f730"} Jul 13 17:12:02.581229 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:02.580852 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" event={"ID":"a269ed30-312c-421b-aa7e-d45f32aab007","Type":"ContainerStarted","Data":"c66b18313780900dd64e3b18e4e2ab63525b393d8fc2abbbc513e3e289cb5a5d"} Jul 13 17:12:02.581229 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:02.581122 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:02.583035 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:02.583014 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" Jul 13 17:12:02.628349 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:02.628290 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf8tz" podStartSLOduration=1.628268969 podStartE2EDuration="1.628268969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:12:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:12:02.620623513 +0000 UTC m=+400.283398702" watchObservedRunningTime="2026-07-13 17:12:02.628268969 +0000 UTC m=+400.291044154" Jul 13 17:12:02.800865 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:02.800833 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh"] Jul 13 17:12:02.801063 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:02.801035 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" podUID="c05b61e5-b79e-4b55-8d79-3f757abd0384" containerName="discovery" containerID="cri-o://362d8bccd019d655a3da79139fe8b92bc23f4f7bfbbf22da7646fcdd3cb361ea" gracePeriod=30 Jul 13 17:12:02.802214 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:02.801794 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:12:03.087599 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.087543 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:12:03.235693 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.235654 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c05b61e5-b79e-4b55-8d79-3f757abd0384-cacerts\") pod \"c05b61e5-b79e-4b55-8d79-3f757abd0384\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " Jul 13 17:12:03.235889 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.235722 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-kubeconfig\") pod \"c05b61e5-b79e-4b55-8d79-3f757abd0384\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " Jul 13 17:12:03.235889 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.235790 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-csr-dns-cert\") pod \"c05b61e5-b79e-4b55-8d79-3f757abd0384\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " Jul 13 17:12:03.235889 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.235812 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c05b61e5-b79e-4b55-8d79-3f757abd0384-local-certs\") pod \"c05b61e5-b79e-4b55-8d79-3f757abd0384\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " Jul 13 17:12:03.235889 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.235857 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-token\") pod \"c05b61e5-b79e-4b55-8d79-3f757abd0384\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " Jul 13 17:12:03.236019 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.235895 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-csr-ca-configmap\") pod \"c05b61e5-b79e-4b55-8d79-3f757abd0384\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " Jul 13 17:12:03.236019 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.235926 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hm669\" (UniqueName: \"kubernetes.io/projected/c05b61e5-b79e-4b55-8d79-3f757abd0384-kube-api-access-hm669\") pod \"c05b61e5-b79e-4b55-8d79-3f757abd0384\" (UID: \"c05b61e5-b79e-4b55-8d79-3f757abd0384\") " Jul 13 17:12:03.236621 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.236564 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "c05b61e5-b79e-4b55-8d79-3f757abd0384" (UID: "c05b61e5-b79e-4b55-8d79-3f757abd0384"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:12:03.239398 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.238823 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c05b61e5-b79e-4b55-8d79-3f757abd0384-cacerts" (OuterVolumeSpecName: "cacerts") pod "c05b61e5-b79e-4b55-8d79-3f757abd0384" (UID: "c05b61e5-b79e-4b55-8d79-3f757abd0384"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:12:03.239398 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.238823 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c05b61e5-b79e-4b55-8d79-3f757abd0384-local-certs" (OuterVolumeSpecName: "local-certs") pod "c05b61e5-b79e-4b55-8d79-3f757abd0384" (UID: "c05b61e5-b79e-4b55-8d79-3f757abd0384"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:12:03.239398 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.238926 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "c05b61e5-b79e-4b55-8d79-3f757abd0384" (UID: "c05b61e5-b79e-4b55-8d79-3f757abd0384"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:12:03.239398 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.238945 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "c05b61e5-b79e-4b55-8d79-3f757abd0384" (UID: "c05b61e5-b79e-4b55-8d79-3f757abd0384"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:12:03.239398 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.239270 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-token" (OuterVolumeSpecName: "istio-token") pod "c05b61e5-b79e-4b55-8d79-3f757abd0384" (UID: "c05b61e5-b79e-4b55-8d79-3f757abd0384"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:12:03.241888 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.241849 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c05b61e5-b79e-4b55-8d79-3f757abd0384-kube-api-access-hm669" (OuterVolumeSpecName: "kube-api-access-hm669") pod "c05b61e5-b79e-4b55-8d79-3f757abd0384" (UID: "c05b61e5-b79e-4b55-8d79-3f757abd0384"). InnerVolumeSpecName "kube-api-access-hm669". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:12:03.336910 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.336869 2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-csr-dns-cert\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:12:03.336910 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.336905 2564 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c05b61e5-b79e-4b55-8d79-3f757abd0384-local-certs\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:12:03.336910 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.336919 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-token\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:12:03.337185 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.336934 2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-csr-ca-configmap\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:12:03.337185 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.336948 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm669\" (UniqueName: \"kubernetes.io/projected/c05b61e5-b79e-4b55-8d79-3f757abd0384-kube-api-access-hm669\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:12:03.337185 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.336960 2564 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c05b61e5-b79e-4b55-8d79-3f757abd0384-cacerts\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:12:03.337185 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.336980 2564 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c05b61e5-b79e-4b55-8d79-3f757abd0384-istio-kubeconfig\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:12:03.588941 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.588842 2564 generic.go:356] "Generic (PLEG): container finished" podID="c05b61e5-b79e-4b55-8d79-3f757abd0384" containerID="362d8bccd019d655a3da79139fe8b92bc23f4f7bfbbf22da7646fcdd3cb361ea" exitCode=0 Jul 13 17:12:03.589320 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.588963 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" Jul 13 17:12:03.589320 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.589010 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" event={"ID":"c05b61e5-b79e-4b55-8d79-3f757abd0384","Type":"ContainerDied","Data":"362d8bccd019d655a3da79139fe8b92bc23f4f7bfbbf22da7646fcdd3cb361ea"} Jul 13 17:12:03.589320 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.589051 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh" event={"ID":"c05b61e5-b79e-4b55-8d79-3f757abd0384","Type":"ContainerDied","Data":"6b55a84578eb400c139fbd7794b149e534befb0425de918573c8537ed16aec2e"} Jul 13 17:12:03.589320 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.589078 2564 scope.go:117] "RemoveContainer" containerID="362d8bccd019d655a3da79139fe8b92bc23f4f7bfbbf22da7646fcdd3cb361ea" Jul 13 17:12:03.623444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.623396 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh"] Jul 13 17:12:03.632797 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:03.632766 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cmxsh"] Jul 13 17:12:04.432319 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.432134 2564 scope.go:117] "RemoveContainer" containerID="362d8bccd019d655a3da79139fe8b92bc23f4f7bfbbf22da7646fcdd3cb361ea" Jul 13 17:12:04.432573 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:12:04.432544 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"362d8bccd019d655a3da79139fe8b92bc23f4f7bfbbf22da7646fcdd3cb361ea\": container with ID starting with 362d8bccd019d655a3da79139fe8b92bc23f4f7bfbbf22da7646fcdd3cb361ea not found: ID does not exist" containerID="362d8bccd019d655a3da79139fe8b92bc23f4f7bfbbf22da7646fcdd3cb361ea" Jul 13 17:12:04.432671 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.432581 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"362d8bccd019d655a3da79139fe8b92bc23f4f7bfbbf22da7646fcdd3cb361ea"} err="failed to get container status \"362d8bccd019d655a3da79139fe8b92bc23f4f7bfbbf22da7646fcdd3cb361ea\": rpc error: code = NotFound desc = could not find container \"362d8bccd019d655a3da79139fe8b92bc23f4f7bfbbf22da7646fcdd3cb361ea\": container with ID starting with 362d8bccd019d655a3da79139fe8b92bc23f4f7bfbbf22da7646fcdd3cb361ea not found: ID does not exist" Jul 13 17:12:04.460386 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.460350 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:12:04.472425 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.472392 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:12:04.472528 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.472462 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:12:04.472528 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.472492 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:12:04.472751 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.472716 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-l5n48"] Jul 13 17:12:04.473086 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.473073 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c05b61e5-b79e-4b55-8d79-3f757abd0384" containerName="discovery" Jul 13 17:12:04.473129 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.473090 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c05b61e5-b79e-4b55-8d79-3f757abd0384" containerName="discovery" Jul 13 17:12:04.473164 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.473157 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c05b61e5-b79e-4b55-8d79-3f757abd0384" containerName="discovery" Jul 13 17:12:04.475442 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.475426 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-l5n48" Jul 13 17:12:04.479921 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.479895 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7nvzq"] Jul 13 17:12:04.481487 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.481411 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-btx7j\"" Jul 13 17:12:04.481590 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.481410 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 13 17:12:04.482700 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.482397 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nvzq" Jul 13 17:12:04.489588 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.489567 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 13 17:12:04.491006 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.490991 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-qwr9w\"" Jul 13 17:12:04.498185 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.498159 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-l5n48"] Jul 13 17:12:04.514334 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.514305 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7nvzq"] Jul 13 17:12:04.549304 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.549275 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/430f9856-48e6-46e5-b4e2-83ce4fcf0e98-cert\") pod \"kserve-controller-manager-d6f4b65b8-l5n48\" (UID: \"430f9856-48e6-46e5-b4e2-83ce4fcf0e98\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-l5n48" Jul 13 17:12:04.549442 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.549342 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tml8d\" (UniqueName: \"kubernetes.io/projected/430f9856-48e6-46e5-b4e2-83ce4fcf0e98-kube-api-access-tml8d\") pod \"kserve-controller-manager-d6f4b65b8-l5n48\" (UID: \"430f9856-48e6-46e5-b4e2-83ce4fcf0e98\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-l5n48" Jul 13 17:12:04.587490 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.587455 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:12:04.597029 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.596987 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" event={"ID":"c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13","Type":"ContainerStarted","Data":"92f3316ba6a6b4301aa22cc83490e383eeeb254062f690ab78c27d143db78588"} Jul 13 17:12:04.626409 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.626353 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" podStartSLOduration=1.5497426 podStartE2EDuration="4.626334427s" totalImagesPullingTime="3.076591827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:12:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:12:04.624621829 +0000 UTC m=+402.287397018" watchObservedRunningTime="2026-07-13 17:12:04.626334427 +0000 UTC m=+402.289109615" Jul 13 17:12:04.650718 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.650669 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8787\" (UniqueName: \"kubernetes.io/projected/ec8ec3a3-9414-48ce-bb17-2344ec6286b8-kube-api-access-p8787\") pod \"llmisvc-controller-manager-777c6bd7d5-7nvzq\" (UID: \"ec8ec3a3-9414-48ce-bb17-2344ec6286b8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nvzq" Jul 13 17:12:04.650913 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.650759 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/430f9856-48e6-46e5-b4e2-83ce4fcf0e98-cert\") pod \"kserve-controller-manager-d6f4b65b8-l5n48\" (UID: \"430f9856-48e6-46e5-b4e2-83ce4fcf0e98\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-l5n48" Jul 13 17:12:04.650913 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.650828 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec8ec3a3-9414-48ce-bb17-2344ec6286b8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7nvzq\" (UID: \"ec8ec3a3-9414-48ce-bb17-2344ec6286b8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nvzq" Jul 13 17:12:04.650913 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.650860 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tml8d\" (UniqueName: \"kubernetes.io/projected/430f9856-48e6-46e5-b4e2-83ce4fcf0e98-kube-api-access-tml8d\") pod \"kserve-controller-manager-d6f4b65b8-l5n48\" (UID: \"430f9856-48e6-46e5-b4e2-83ce4fcf0e98\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-l5n48" Jul 13 17:12:04.653400 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.653376 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/430f9856-48e6-46e5-b4e2-83ce4fcf0e98-cert\") pod \"kserve-controller-manager-d6f4b65b8-l5n48\" (UID: \"430f9856-48e6-46e5-b4e2-83ce4fcf0e98\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-l5n48" Jul 13 17:12:04.662474 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.662440 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tml8d\" (UniqueName: \"kubernetes.io/projected/430f9856-48e6-46e5-b4e2-83ce4fcf0e98-kube-api-access-tml8d\") pod \"kserve-controller-manager-d6f4b65b8-l5n48\" (UID: \"430f9856-48e6-46e5-b4e2-83ce4fcf0e98\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-l5n48" Jul 13 17:12:04.663994 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.663935 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:12:04.751722 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.751590 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8787\" (UniqueName: \"kubernetes.io/projected/ec8ec3a3-9414-48ce-bb17-2344ec6286b8-kube-api-access-p8787\") pod \"llmisvc-controller-manager-777c6bd7d5-7nvzq\" (UID: \"ec8ec3a3-9414-48ce-bb17-2344ec6286b8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nvzq" Jul 13 17:12:04.752141 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.752086 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec8ec3a3-9414-48ce-bb17-2344ec6286b8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7nvzq\" (UID: \"ec8ec3a3-9414-48ce-bb17-2344ec6286b8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nvzq" Jul 13 17:12:04.756526 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.756495 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec8ec3a3-9414-48ce-bb17-2344ec6286b8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7nvzq\" (UID: \"ec8ec3a3-9414-48ce-bb17-2344ec6286b8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nvzq" Jul 13 17:12:04.763591 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.763565 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8787\" (UniqueName: \"kubernetes.io/projected/ec8ec3a3-9414-48ce-bb17-2344ec6286b8-kube-api-access-p8787\") pod \"llmisvc-controller-manager-777c6bd7d5-7nvzq\" (UID: \"ec8ec3a3-9414-48ce-bb17-2344ec6286b8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nvzq" Jul 13 17:12:04.787388 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.787357 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-l5n48" Jul 13 17:12:04.813202 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.813169 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nvzq" Jul 13 17:12:04.962476 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.962438 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c05b61e5-b79e-4b55-8d79-3f757abd0384" path="/var/lib/kubelet/pods/c05b61e5-b79e-4b55-8d79-3f757abd0384/volumes" Jul 13 17:12:04.985954 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:04.985925 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-l5n48"] Jul 13 17:12:04.988012 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:12:04.987985 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod430f9856_48e6_46e5_b4e2_83ce4fcf0e98.slice/crio-e947a7f530792def736f5c89eb08395344e3c4c66ae7cf2de399913ddaadb9ac WatchSource:0}: Error finding container e947a7f530792def736f5c89eb08395344e3c4c66ae7cf2de399913ddaadb9ac: Status 404 returned error can't find the container with id e947a7f530792def736f5c89eb08395344e3c4c66ae7cf2de399913ddaadb9ac Jul 13 17:12:05.025758 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:05.022191 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7nvzq"] Jul 13 17:12:05.030165 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:12:05.030141 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podec8ec3a3_9414_48ce_bb17_2344ec6286b8.slice/crio-83a5ee5a6374ff70f76d4c938091bd6fb40792681bd73c10cbeb3499247df5f1 WatchSource:0}: Error finding container 83a5ee5a6374ff70f76d4c938091bd6fb40792681bd73c10cbeb3499247df5f1: Status 404 returned error can't find the container with id 83a5ee5a6374ff70f76d4c938091bd6fb40792681bd73c10cbeb3499247df5f1 Jul 13 17:12:05.238704 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:05.238663 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:05.243091 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:05.243066 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:05.603032 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:05.602988 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nvzq" event={"ID":"ec8ec3a3-9414-48ce-bb17-2344ec6286b8","Type":"ContainerStarted","Data":"83a5ee5a6374ff70f76d4c938091bd6fb40792681bd73c10cbeb3499247df5f1"} Jul 13 17:12:05.604428 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:05.604396 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-l5n48" event={"ID":"430f9856-48e6-46e5-b4e2-83ce4fcf0e98","Type":"ContainerStarted","Data":"e947a7f530792def736f5c89eb08395344e3c4c66ae7cf2de399913ddaadb9ac"} Jul 13 17:12:05.604706 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:05.604688 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:05.605759 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:05.605723 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf449pjx" Jul 13 17:12:05.770069 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:05.770037 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:12:05.869398 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:05.869323 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:12:08.555000 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:08.554950 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-7hstg" Jul 13 17:12:09.626362 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:09.626325 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nvzq" event={"ID":"ec8ec3a3-9414-48ce-bb17-2344ec6286b8","Type":"ContainerStarted","Data":"85afeb76539a6846626b876441555bbed8be4377b8246009d50cb89167479a75"} Jul 13 17:12:09.626873 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:09.626389 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nvzq" Jul 13 17:12:09.627879 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:09.627851 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-l5n48" event={"ID":"430f9856-48e6-46e5-b4e2-83ce4fcf0e98","Type":"ContainerStarted","Data":"727b215f09f1894c95772774dfd3544b40e8a34ad2c515901641dc88c0379c41"} Jul 13 17:12:09.628003 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:09.627990 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-l5n48" Jul 13 17:12:09.645635 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:09.645575 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nvzq" podStartSLOduration=1.954640941 podStartE2EDuration="5.64555871s" totalImagesPullingTime="3.690917769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:12:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:12:09.644605984 +0000 UTC m=+407.307381173" watchObservedRunningTime="2026-07-13 17:12:09.64555871 +0000 UTC m=+407.308333898" Jul 13 17:12:09.665066 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:09.665005 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-l5n48" podStartSLOduration=1.93393832 podStartE2EDuration="5.664987521s" totalImagesPullingTime="3.731049201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:12:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:12:09.66313078 +0000 UTC m=+407.325905969" watchObservedRunningTime="2026-07-13 17:12:09.664987521 +0000 UTC m=+407.327762716" Jul 13 17:12:10.563640 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:10.563596 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-9j2b6" Jul 13 17:12:40.636045 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:40.636008 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nvzq" Jul 13 17:12:40.639530 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:40.639504 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-l5n48" Jul 13 17:12:53.117670 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:53.117604 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:12:53.215057 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:53.215023 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:12:53.324952 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:53.324906 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:12:54.111265 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:54.111231 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:12:54.256054 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:12:54.256025 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:05.730058 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.729989 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5978bd6596-pgchn"] Jul 13 17:13:05.732679 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.732660 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.743871 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.743841 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5978bd6596-pgchn"] Jul 13 17:13:05.786144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.786110 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:05.851183 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.851141 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:05.886448 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.886418 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da10a14e-1b38-40a1-b736-4c20b1862f71-service-ca\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.886630 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.886455 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da10a14e-1b38-40a1-b736-4c20b1862f71-trusted-ca-bundle\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.886630 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.886476 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da10a14e-1b38-40a1-b736-4c20b1862f71-console-oauth-config\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.886630 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.886529 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2lc2\" (UniqueName: \"kubernetes.io/projected/da10a14e-1b38-40a1-b736-4c20b1862f71-kube-api-access-f2lc2\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.886630 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.886607 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da10a14e-1b38-40a1-b736-4c20b1862f71-oauth-serving-cert\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.886844 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.886632 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da10a14e-1b38-40a1-b736-4c20b1862f71-console-config\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.886844 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.886657 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da10a14e-1b38-40a1-b736-4c20b1862f71-console-serving-cert\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.987603 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.987504 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da10a14e-1b38-40a1-b736-4c20b1862f71-service-ca\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.987603 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.987586 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da10a14e-1b38-40a1-b736-4c20b1862f71-trusted-ca-bundle\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.987603 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.987607 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da10a14e-1b38-40a1-b736-4c20b1862f71-console-oauth-config\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.987949 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.987635 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2lc2\" (UniqueName: \"kubernetes.io/projected/da10a14e-1b38-40a1-b736-4c20b1862f71-kube-api-access-f2lc2\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.987949 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.987661 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da10a14e-1b38-40a1-b736-4c20b1862f71-oauth-serving-cert\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.987949 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.987790 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da10a14e-1b38-40a1-b736-4c20b1862f71-console-config\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.987949 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.987851 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da10a14e-1b38-40a1-b736-4c20b1862f71-console-serving-cert\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.988284 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.988255 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da10a14e-1b38-40a1-b736-4c20b1862f71-service-ca\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.988466 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.988446 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da10a14e-1b38-40a1-b736-4c20b1862f71-oauth-serving-cert\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.988546 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.988524 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da10a14e-1b38-40a1-b736-4c20b1862f71-trusted-ca-bundle\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.988747 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.988710 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da10a14e-1b38-40a1-b736-4c20b1862f71-console-config\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.990432 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.990411 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da10a14e-1b38-40a1-b736-4c20b1862f71-console-oauth-config\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.990487 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.990411 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da10a14e-1b38-40a1-b736-4c20b1862f71-console-serving-cert\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:05.997300 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:05.997244 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2lc2\" (UniqueName: \"kubernetes.io/projected/da10a14e-1b38-40a1-b736-4c20b1862f71-kube-api-access-f2lc2\") pod \"console-5978bd6596-pgchn\" (UID: \"da10a14e-1b38-40a1-b736-4c20b1862f71\") " pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:06.042338 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:06.042299 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:06.179532 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:06.179503 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5978bd6596-pgchn"] Jul 13 17:13:06.181750 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:13:06.181696 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda10a14e_1b38_40a1_b736_4c20b1862f71.slice/crio-f1a848cbaa8d9d647a79d507d29523eb8fe33b02fe9c63080164fb91b97ef767 WatchSource:0}: Error finding container f1a848cbaa8d9d647a79d507d29523eb8fe33b02fe9c63080164fb91b97ef767: Status 404 returned error can't find the container with id f1a848cbaa8d9d647a79d507d29523eb8fe33b02fe9c63080164fb91b97ef767 Jul 13 17:13:06.890952 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:06.890915 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5978bd6596-pgchn" event={"ID":"da10a14e-1b38-40a1-b736-4c20b1862f71","Type":"ContainerStarted","Data":"0fa7eeaed06785eefa411d806aae5e6b6e2e670f84af9511de8a9adb39152192"} Jul 13 17:13:06.891352 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:06.890966 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5978bd6596-pgchn" event={"ID":"da10a14e-1b38-40a1-b736-4c20b1862f71","Type":"ContainerStarted","Data":"f1a848cbaa8d9d647a79d507d29523eb8fe33b02fe9c63080164fb91b97ef767"} Jul 13 17:13:06.911035 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:06.910984 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5978bd6596-pgchn" podStartSLOduration=1.910968212 podStartE2EDuration="1.910968212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:13:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:13:06.908958534 +0000 UTC m=+464.571733721" watchObservedRunningTime="2026-07-13 17:13:06.910968212 +0000 UTC m=+464.573743400" Jul 13 17:13:09.467200 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:09.467152 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx"] Jul 13 17:13:09.470222 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:09.470203 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx" Jul 13 17:13:09.473250 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:09.473226 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 17:13:09.473335 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:09.473252 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 17:13:09.474491 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:09.474474 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hzghr\"" Jul 13 17:13:09.478697 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:09.478675 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx"] Jul 13 17:13:09.622663 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:09.622611 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99f3124a-f8f6-403b-9984-b4aacf9e5e74-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx\" (UID: \"99f3124a-f8f6-403b-9984-b4aacf9e5e74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx" Jul 13 17:13:09.622868 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:09.622699 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzd9w\" (UniqueName: \"kubernetes.io/projected/99f3124a-f8f6-403b-9984-b4aacf9e5e74-kube-api-access-nzd9w\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx\" (UID: \"99f3124a-f8f6-403b-9984-b4aacf9e5e74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx" Jul 13 17:13:09.622868 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:09.622816 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99f3124a-f8f6-403b-9984-b4aacf9e5e74-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx\" (UID: \"99f3124a-f8f6-403b-9984-b4aacf9e5e74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx" Jul 13 17:13:09.724170 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:09.724077 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzd9w\" (UniqueName: \"kubernetes.io/projected/99f3124a-f8f6-403b-9984-b4aacf9e5e74-kube-api-access-nzd9w\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx\" (UID: \"99f3124a-f8f6-403b-9984-b4aacf9e5e74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx" Jul 13 17:13:09.724335 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:09.724168 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99f3124a-f8f6-403b-9984-b4aacf9e5e74-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx\" (UID: \"99f3124a-f8f6-403b-9984-b4aacf9e5e74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx" Jul 13 17:13:09.724335 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:09.724224 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99f3124a-f8f6-403b-9984-b4aacf9e5e74-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx\" (UID: \"99f3124a-f8f6-403b-9984-b4aacf9e5e74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx" Jul 13 17:13:09.724578 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:09.724557 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99f3124a-f8f6-403b-9984-b4aacf9e5e74-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx\" (UID: \"99f3124a-f8f6-403b-9984-b4aacf9e5e74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx" Jul 13 17:13:09.724638 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:09.724594 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99f3124a-f8f6-403b-9984-b4aacf9e5e74-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx\" (UID: \"99f3124a-f8f6-403b-9984-b4aacf9e5e74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx" Jul 13 17:13:09.732363 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:09.732335 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzd9w\" (UniqueName: \"kubernetes.io/projected/99f3124a-f8f6-403b-9984-b4aacf9e5e74-kube-api-access-nzd9w\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx\" (UID: \"99f3124a-f8f6-403b-9984-b4aacf9e5e74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx" Jul 13 17:13:09.781575 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:09.781541 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx" Jul 13 17:13:09.922197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:09.922172 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx"] Jul 13 17:13:09.924789 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:13:09.924761 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99f3124a_f8f6_403b_9984_b4aacf9e5e74.slice/crio-44d52161b09dcfaaa37c52f719dc49d2cc307dc62d076160bbb2d6b0fabf99f0 WatchSource:0}: Error finding container 44d52161b09dcfaaa37c52f719dc49d2cc307dc62d076160bbb2d6b0fabf99f0: Status 404 returned error can't find the container with id 44d52161b09dcfaaa37c52f719dc49d2cc307dc62d076160bbb2d6b0fabf99f0 Jul 13 17:13:10.051168 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.051137 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb"] Jul 13 17:13:10.053992 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.053972 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb" Jul 13 17:13:10.060552 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.060527 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb"] Jul 13 17:13:10.228857 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.228759 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhfrx\" (UniqueName: \"kubernetes.io/projected/3236fc6b-d795-48f4-8298-2866758c75b5-kube-api-access-xhfrx\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb\" (UID: \"3236fc6b-d795-48f4-8298-2866758c75b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb" Jul 13 17:13:10.228857 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.228808 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3236fc6b-d795-48f4-8298-2866758c75b5-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb\" (UID: \"3236fc6b-d795-48f4-8298-2866758c75b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb" Jul 13 17:13:10.228857 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.228830 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3236fc6b-d795-48f4-8298-2866758c75b5-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb\" (UID: \"3236fc6b-d795-48f4-8298-2866758c75b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb" Jul 13 17:13:10.329470 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.329434 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhfrx\" (UniqueName: \"kubernetes.io/projected/3236fc6b-d795-48f4-8298-2866758c75b5-kube-api-access-xhfrx\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb\" (UID: \"3236fc6b-d795-48f4-8298-2866758c75b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb" Jul 13 17:13:10.329634 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.329484 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3236fc6b-d795-48f4-8298-2866758c75b5-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb\" (UID: \"3236fc6b-d795-48f4-8298-2866758c75b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb" Jul 13 17:13:10.329634 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.329504 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3236fc6b-d795-48f4-8298-2866758c75b5-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb\" (UID: \"3236fc6b-d795-48f4-8298-2866758c75b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb" Jul 13 17:13:10.329938 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.329917 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3236fc6b-d795-48f4-8298-2866758c75b5-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb\" (UID: \"3236fc6b-d795-48f4-8298-2866758c75b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb" Jul 13 17:13:10.329990 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.329932 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3236fc6b-d795-48f4-8298-2866758c75b5-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb\" (UID: \"3236fc6b-d795-48f4-8298-2866758c75b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb" Jul 13 17:13:10.340987 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.340961 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhfrx\" (UniqueName: \"kubernetes.io/projected/3236fc6b-d795-48f4-8298-2866758c75b5-kube-api-access-xhfrx\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb\" (UID: \"3236fc6b-d795-48f4-8298-2866758c75b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb" Jul 13 17:13:10.397505 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.397460 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb" Jul 13 17:13:10.533103 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.533077 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb"] Jul 13 17:13:10.534872 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:13:10.534843 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3236fc6b_d795_48f4_8298_2866758c75b5.slice/crio-b9eaf40264f2574b71fc5e9c3be4e3ef053aa7acfc7b4337b5afa8c071ef0467 WatchSource:0}: Error finding container b9eaf40264f2574b71fc5e9c3be4e3ef053aa7acfc7b4337b5afa8c071ef0467: Status 404 returned error can't find the container with id b9eaf40264f2574b71fc5e9c3be4e3ef053aa7acfc7b4337b5afa8c071ef0467 Jul 13 17:13:10.537174 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.537151 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2"] Jul 13 17:13:10.541138 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.541122 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2" Jul 13 17:13:10.547453 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.547430 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2"] Jul 13 17:13:10.732843 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.732794 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7lvq\" (UniqueName: \"kubernetes.io/projected/2e440e1a-c3c6-4903-88d5-fc70b40a4c17-kube-api-access-c7lvq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2\" (UID: \"2e440e1a-c3c6-4903-88d5-fc70b40a4c17\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2" Jul 13 17:13:10.733017 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.732914 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e440e1a-c3c6-4903-88d5-fc70b40a4c17-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2\" (UID: \"2e440e1a-c3c6-4903-88d5-fc70b40a4c17\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2" Jul 13 17:13:10.733066 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.733043 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e440e1a-c3c6-4903-88d5-fc70b40a4c17-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2\" (UID: \"2e440e1a-c3c6-4903-88d5-fc70b40a4c17\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2" Jul 13 17:13:10.834344 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.834250 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7lvq\" (UniqueName: \"kubernetes.io/projected/2e440e1a-c3c6-4903-88d5-fc70b40a4c17-kube-api-access-c7lvq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2\" (UID: \"2e440e1a-c3c6-4903-88d5-fc70b40a4c17\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2" Jul 13 17:13:10.834344 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.834324 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e440e1a-c3c6-4903-88d5-fc70b40a4c17-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2\" (UID: \"2e440e1a-c3c6-4903-88d5-fc70b40a4c17\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2" Jul 13 17:13:10.834541 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.834400 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e440e1a-c3c6-4903-88d5-fc70b40a4c17-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2\" (UID: \"2e440e1a-c3c6-4903-88d5-fc70b40a4c17\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2" Jul 13 17:13:10.834787 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.834764 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e440e1a-c3c6-4903-88d5-fc70b40a4c17-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2\" (UID: \"2e440e1a-c3c6-4903-88d5-fc70b40a4c17\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2" Jul 13 17:13:10.834832 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.834786 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e440e1a-c3c6-4903-88d5-fc70b40a4c17-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2\" (UID: \"2e440e1a-c3c6-4903-88d5-fc70b40a4c17\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2" Jul 13 17:13:10.845598 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.845558 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7lvq\" (UniqueName: \"kubernetes.io/projected/2e440e1a-c3c6-4903-88d5-fc70b40a4c17-kube-api-access-c7lvq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2\" (UID: \"2e440e1a-c3c6-4903-88d5-fc70b40a4c17\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2" Jul 13 17:13:10.852399 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.852371 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2" Jul 13 17:13:10.868989 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.868948 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs"] Jul 13 17:13:10.871887 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.871865 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs" Jul 13 17:13:10.881528 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.881500 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs"] Jul 13 17:13:10.913070 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.913034 2564 generic.go:356] "Generic (PLEG): container finished" podID="99f3124a-f8f6-403b-9984-b4aacf9e5e74" containerID="7e698f4223f9eb2346b352e23d37a16291a900712adcbe2e01173c45cbabdd0f" exitCode=0 Jul 13 17:13:10.913243 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.913125 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx" event={"ID":"99f3124a-f8f6-403b-9984-b4aacf9e5e74","Type":"ContainerDied","Data":"7e698f4223f9eb2346b352e23d37a16291a900712adcbe2e01173c45cbabdd0f"} Jul 13 17:13:10.913243 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.913192 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx" event={"ID":"99f3124a-f8f6-403b-9984-b4aacf9e5e74","Type":"ContainerStarted","Data":"44d52161b09dcfaaa37c52f719dc49d2cc307dc62d076160bbb2d6b0fabf99f0"} Jul 13 17:13:10.915109 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.915083 2564 generic.go:356] "Generic (PLEG): container finished" podID="3236fc6b-d795-48f4-8298-2866758c75b5" containerID="125ed3dee33ec881c1cf5a913753407bdff4e83a6798b80fde2476ff1f033665" exitCode=0 Jul 13 17:13:10.915213 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.915196 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb" event={"ID":"3236fc6b-d795-48f4-8298-2866758c75b5","Type":"ContainerDied","Data":"125ed3dee33ec881c1cf5a913753407bdff4e83a6798b80fde2476ff1f033665"} Jul 13 17:13:10.915262 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:10.915236 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb" event={"ID":"3236fc6b-d795-48f4-8298-2866758c75b5","Type":"ContainerStarted","Data":"b9eaf40264f2574b71fc5e9c3be4e3ef053aa7acfc7b4337b5afa8c071ef0467"} Jul 13 17:13:11.001181 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.001155 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2"] Jul 13 17:13:11.003211 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:13:11.003168 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e440e1a_c3c6_4903_88d5_fc70b40a4c17.slice/crio-c74cf9ab9a95b3ed228290bb3679cc4ab4f8021bb159f5e19eaf5a353510a966 WatchSource:0}: Error finding container c74cf9ab9a95b3ed228290bb3679cc4ab4f8021bb159f5e19eaf5a353510a966: Status 404 returned error can't find the container with id c74cf9ab9a95b3ed228290bb3679cc4ab4f8021bb159f5e19eaf5a353510a966 Jul 13 17:13:11.035777 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.035717 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d3aec72-80db-4422-a290-45527c4d9c6d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs\" (UID: \"9d3aec72-80db-4422-a290-45527c4d9c6d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs" Jul 13 17:13:11.035893 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.035799 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfw8x\" (UniqueName: \"kubernetes.io/projected/9d3aec72-80db-4422-a290-45527c4d9c6d-kube-api-access-mfw8x\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs\" (UID: \"9d3aec72-80db-4422-a290-45527c4d9c6d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs" Jul 13 17:13:11.035893 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.035867 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d3aec72-80db-4422-a290-45527c4d9c6d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs\" (UID: \"9d3aec72-80db-4422-a290-45527c4d9c6d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs" Jul 13 17:13:11.137042 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.137001 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d3aec72-80db-4422-a290-45527c4d9c6d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs\" (UID: \"9d3aec72-80db-4422-a290-45527c4d9c6d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs" Jul 13 17:13:11.137228 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.137096 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d3aec72-80db-4422-a290-45527c4d9c6d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs\" (UID: \"9d3aec72-80db-4422-a290-45527c4d9c6d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs" Jul 13 17:13:11.137228 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.137120 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfw8x\" (UniqueName: \"kubernetes.io/projected/9d3aec72-80db-4422-a290-45527c4d9c6d-kube-api-access-mfw8x\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs\" (UID: \"9d3aec72-80db-4422-a290-45527c4d9c6d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs" Jul 13 17:13:11.137443 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.137420 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d3aec72-80db-4422-a290-45527c4d9c6d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs\" (UID: \"9d3aec72-80db-4422-a290-45527c4d9c6d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs" Jul 13 17:13:11.137505 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.137441 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d3aec72-80db-4422-a290-45527c4d9c6d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs\" (UID: \"9d3aec72-80db-4422-a290-45527c4d9c6d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs" Jul 13 17:13:11.146599 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.146577 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfw8x\" (UniqueName: \"kubernetes.io/projected/9d3aec72-80db-4422-a290-45527c4d9c6d-kube-api-access-mfw8x\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs\" (UID: \"9d3aec72-80db-4422-a290-45527c4d9c6d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs" Jul 13 17:13:11.183520 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.183481 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs" Jul 13 17:13:11.317289 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.317262 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs"] Jul 13 17:13:11.318665 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:13:11.318631 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d3aec72_80db_4422_a290_45527c4d9c6d.slice/crio-cfe76f9955e24d2632778b07585f5f8dcaf2df87e1622065393aae6ad0e54d8a WatchSource:0}: Error finding container cfe76f9955e24d2632778b07585f5f8dcaf2df87e1622065393aae6ad0e54d8a: Status 404 returned error can't find the container with id cfe76f9955e24d2632778b07585f5f8dcaf2df87e1622065393aae6ad0e54d8a Jul 13 17:13:11.923749 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.923703 2564 generic.go:356] "Generic (PLEG): container finished" podID="2e440e1a-c3c6-4903-88d5-fc70b40a4c17" containerID="004d67572d13bfb7af331563f55e37a15bfcf0dcfbf1db988832101485cd1e28" exitCode=0 Jul 13 17:13:11.924150 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.923763 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2" event={"ID":"2e440e1a-c3c6-4903-88d5-fc70b40a4c17","Type":"ContainerDied","Data":"004d67572d13bfb7af331563f55e37a15bfcf0dcfbf1db988832101485cd1e28"} Jul 13 17:13:11.924150 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.923814 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2" event={"ID":"2e440e1a-c3c6-4903-88d5-fc70b40a4c17","Type":"ContainerStarted","Data":"c74cf9ab9a95b3ed228290bb3679cc4ab4f8021bb159f5e19eaf5a353510a966"} Jul 13 17:13:11.925776 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.925745 2564 generic.go:356] "Generic (PLEG): container finished" podID="3236fc6b-d795-48f4-8298-2866758c75b5" containerID="2a911eff273a76c7d2a023adcfd1a8f4525f7303c4d6022f69357fe80b5e2af4" exitCode=0 Jul 13 17:13:11.925937 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.925887 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb" event={"ID":"3236fc6b-d795-48f4-8298-2866758c75b5","Type":"ContainerDied","Data":"2a911eff273a76c7d2a023adcfd1a8f4525f7303c4d6022f69357fe80b5e2af4"} Jul 13 17:13:11.927491 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.927463 2564 generic.go:356] "Generic (PLEG): container finished" podID="9d3aec72-80db-4422-a290-45527c4d9c6d" containerID="a463561aae519b8c6af3248a17c1261d8a86ffac5d04a1e939e06c446b5f53f4" exitCode=0 Jul 13 17:13:11.927610 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.927496 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs" event={"ID":"9d3aec72-80db-4422-a290-45527c4d9c6d","Type":"ContainerDied","Data":"a463561aae519b8c6af3248a17c1261d8a86ffac5d04a1e939e06c446b5f53f4"} Jul 13 17:13:11.927610 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:11.927546 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs" event={"ID":"9d3aec72-80db-4422-a290-45527c4d9c6d","Type":"ContainerStarted","Data":"cfe76f9955e24d2632778b07585f5f8dcaf2df87e1622065393aae6ad0e54d8a"} Jul 13 17:13:12.814223 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:12.814193 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:12.934341 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:12.934266 2564 generic.go:356] "Generic (PLEG): container finished" podID="3236fc6b-d795-48f4-8298-2866758c75b5" containerID="2b9a81ddb62247edab0d097b654f6a602c65f587c55ae08bb343f719956a8d87" exitCode=0 Jul 13 17:13:12.934341 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:12.934327 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb" event={"ID":"3236fc6b-d795-48f4-8298-2866758c75b5","Type":"ContainerDied","Data":"2b9a81ddb62247edab0d097b654f6a602c65f587c55ae08bb343f719956a8d87"} Jul 13 17:13:12.936176 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:12.936149 2564 generic.go:356] "Generic (PLEG): container finished" podID="99f3124a-f8f6-403b-9984-b4aacf9e5e74" containerID="f968bd68e27335c9bcdb7a24edabf94a15cabcf23681bf7045c97315d62683f5" exitCode=0 Jul 13 17:13:12.936296 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:12.936226 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx" event={"ID":"99f3124a-f8f6-403b-9984-b4aacf9e5e74","Type":"ContainerDied","Data":"f968bd68e27335c9bcdb7a24edabf94a15cabcf23681bf7045c97315d62683f5"} Jul 13 17:13:12.938139 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:12.938113 2564 generic.go:356] "Generic (PLEG): container finished" podID="2e440e1a-c3c6-4903-88d5-fc70b40a4c17" containerID="72d9b732f6c8a394f3c7ea5eeff6657fe6bde8f6ebc9e3d671f4469183507aeb" exitCode=0 Jul 13 17:13:12.938237 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:12.938151 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2" event={"ID":"2e440e1a-c3c6-4903-88d5-fc70b40a4c17","Type":"ContainerDied","Data":"72d9b732f6c8a394f3c7ea5eeff6657fe6bde8f6ebc9e3d671f4469183507aeb"} Jul 13 17:13:13.871298 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:13.871257 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:13.944370 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:13.944327 2564 generic.go:356] "Generic (PLEG): container finished" podID="9d3aec72-80db-4422-a290-45527c4d9c6d" containerID="3bd67e80a3e6ff1d522035859e9156d4a5b2cddd4df90421f46e4d1a40ae0f8b" exitCode=0 Jul 13 17:13:13.944842 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:13.944404 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs" event={"ID":"9d3aec72-80db-4422-a290-45527c4d9c6d","Type":"ContainerDied","Data":"3bd67e80a3e6ff1d522035859e9156d4a5b2cddd4df90421f46e4d1a40ae0f8b"} Jul 13 17:13:13.946864 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:13.946840 2564 generic.go:356] "Generic (PLEG): container finished" podID="99f3124a-f8f6-403b-9984-b4aacf9e5e74" containerID="feab07c43132a97e77d296924ce9eed284d94b78512b2248353ce93d2bbe70e4" exitCode=0 Jul 13 17:13:13.946964 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:13.946918 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx" event={"ID":"99f3124a-f8f6-403b-9984-b4aacf9e5e74","Type":"ContainerDied","Data":"feab07c43132a97e77d296924ce9eed284d94b78512b2248353ce93d2bbe70e4"} Jul 13 17:13:13.955412 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:13.950118 2564 generic.go:356] "Generic (PLEG): container finished" podID="2e440e1a-c3c6-4903-88d5-fc70b40a4c17" containerID="d41c039afb30dfddfb6dc9f73e30b65ceaa7a35492ad865bde316bd77a4ae0ca" exitCode=0 Jul 13 17:13:13.955412 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:13.950575 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2" event={"ID":"2e440e1a-c3c6-4903-88d5-fc70b40a4c17","Type":"ContainerDied","Data":"d41c039afb30dfddfb6dc9f73e30b65ceaa7a35492ad865bde316bd77a4ae0ca"} Jul 13 17:13:14.100122 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:14.100098 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb" Jul 13 17:13:14.157867 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:14.157832 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3236fc6b-d795-48f4-8298-2866758c75b5-bundle\") pod \"3236fc6b-d795-48f4-8298-2866758c75b5\" (UID: \"3236fc6b-d795-48f4-8298-2866758c75b5\") " Jul 13 17:13:14.157867 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:14.157878 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhfrx\" (UniqueName: \"kubernetes.io/projected/3236fc6b-d795-48f4-8298-2866758c75b5-kube-api-access-xhfrx\") pod \"3236fc6b-d795-48f4-8298-2866758c75b5\" (UID: \"3236fc6b-d795-48f4-8298-2866758c75b5\") " Jul 13 17:13:14.158103 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:14.157926 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3236fc6b-d795-48f4-8298-2866758c75b5-util\") pod \"3236fc6b-d795-48f4-8298-2866758c75b5\" (UID: \"3236fc6b-d795-48f4-8298-2866758c75b5\") " Jul 13 17:13:14.158427 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:14.158399 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3236fc6b-d795-48f4-8298-2866758c75b5-bundle" (OuterVolumeSpecName: "bundle") pod "3236fc6b-d795-48f4-8298-2866758c75b5" (UID: "3236fc6b-d795-48f4-8298-2866758c75b5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:13:14.160336 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:14.160312 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3236fc6b-d795-48f4-8298-2866758c75b5-kube-api-access-xhfrx" (OuterVolumeSpecName: "kube-api-access-xhfrx") pod "3236fc6b-d795-48f4-8298-2866758c75b5" (UID: "3236fc6b-d795-48f4-8298-2866758c75b5"). InnerVolumeSpecName "kube-api-access-xhfrx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:13:14.163300 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:14.163273 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3236fc6b-d795-48f4-8298-2866758c75b5-util" (OuterVolumeSpecName: "util") pod "3236fc6b-d795-48f4-8298-2866758c75b5" (UID: "3236fc6b-d795-48f4-8298-2866758c75b5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:13:14.259007 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:14.258908 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3236fc6b-d795-48f4-8298-2866758c75b5-bundle\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:14.259007 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:14.258946 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhfrx\" (UniqueName: \"kubernetes.io/projected/3236fc6b-d795-48f4-8298-2866758c75b5-kube-api-access-xhfrx\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:14.259007 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:14.258958 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3236fc6b-d795-48f4-8298-2866758c75b5-util\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:14.957326 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:14.957290 2564 generic.go:356] "Generic (PLEG): container finished" podID="9d3aec72-80db-4422-a290-45527c4d9c6d" containerID="e405d3c815d9c5d983b207efb7f1ee108435135f06af6a84422f97456d48622c" exitCode=0 Jul 13 17:13:14.959485 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:14.959460 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb" Jul 13 17:13:14.961645 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:14.961619 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs" event={"ID":"9d3aec72-80db-4422-a290-45527c4d9c6d","Type":"ContainerDied","Data":"e405d3c815d9c5d983b207efb7f1ee108435135f06af6a84422f97456d48622c"} Jul 13 17:13:14.961788 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:14.961670 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb" event={"ID":"3236fc6b-d795-48f4-8298-2866758c75b5","Type":"ContainerDied","Data":"b9eaf40264f2574b71fc5e9c3be4e3ef053aa7acfc7b4337b5afa8c071ef0467"} Jul 13 17:13:14.961788 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:14.961695 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9eaf40264f2574b71fc5e9c3be4e3ef053aa7acfc7b4337b5afa8c071ef0467" Jul 13 17:13:15.095932 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.095909 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2" Jul 13 17:13:15.133167 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.133143 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx" Jul 13 17:13:15.167633 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.167604 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e440e1a-c3c6-4903-88d5-fc70b40a4c17-util\") pod \"2e440e1a-c3c6-4903-88d5-fc70b40a4c17\" (UID: \"2e440e1a-c3c6-4903-88d5-fc70b40a4c17\") " Jul 13 17:13:15.167633 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.167645 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7lvq\" (UniqueName: \"kubernetes.io/projected/2e440e1a-c3c6-4903-88d5-fc70b40a4c17-kube-api-access-c7lvq\") pod \"2e440e1a-c3c6-4903-88d5-fc70b40a4c17\" (UID: \"2e440e1a-c3c6-4903-88d5-fc70b40a4c17\") " Jul 13 17:13:15.167893 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.167678 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99f3124a-f8f6-403b-9984-b4aacf9e5e74-util\") pod \"99f3124a-f8f6-403b-9984-b4aacf9e5e74\" (UID: \"99f3124a-f8f6-403b-9984-b4aacf9e5e74\") " Jul 13 17:13:15.167893 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.167707 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e440e1a-c3c6-4903-88d5-fc70b40a4c17-bundle\") pod \"2e440e1a-c3c6-4903-88d5-fc70b40a4c17\" (UID: \"2e440e1a-c3c6-4903-88d5-fc70b40a4c17\") " Jul 13 17:13:15.167893 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.167761 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99f3124a-f8f6-403b-9984-b4aacf9e5e74-bundle\") pod \"99f3124a-f8f6-403b-9984-b4aacf9e5e74\" (UID: \"99f3124a-f8f6-403b-9984-b4aacf9e5e74\") " Jul 13 17:13:15.167893 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.167811 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzd9w\" (UniqueName: \"kubernetes.io/projected/99f3124a-f8f6-403b-9984-b4aacf9e5e74-kube-api-access-nzd9w\") pod \"99f3124a-f8f6-403b-9984-b4aacf9e5e74\" (UID: \"99f3124a-f8f6-403b-9984-b4aacf9e5e74\") " Jul 13 17:13:15.168378 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.168348 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f3124a-f8f6-403b-9984-b4aacf9e5e74-bundle" (OuterVolumeSpecName: "bundle") pod "99f3124a-f8f6-403b-9984-b4aacf9e5e74" (UID: "99f3124a-f8f6-403b-9984-b4aacf9e5e74"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:13:15.168478 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.168446 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e440e1a-c3c6-4903-88d5-fc70b40a4c17-bundle" (OuterVolumeSpecName: "bundle") pod "2e440e1a-c3c6-4903-88d5-fc70b40a4c17" (UID: "2e440e1a-c3c6-4903-88d5-fc70b40a4c17"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:13:15.170480 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.170455 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e440e1a-c3c6-4903-88d5-fc70b40a4c17-kube-api-access-c7lvq" (OuterVolumeSpecName: "kube-api-access-c7lvq") pod "2e440e1a-c3c6-4903-88d5-fc70b40a4c17" (UID: "2e440e1a-c3c6-4903-88d5-fc70b40a4c17"). InnerVolumeSpecName "kube-api-access-c7lvq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:13:15.170691 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.170662 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99f3124a-f8f6-403b-9984-b4aacf9e5e74-kube-api-access-nzd9w" (OuterVolumeSpecName: "kube-api-access-nzd9w") pod "99f3124a-f8f6-403b-9984-b4aacf9e5e74" (UID: "99f3124a-f8f6-403b-9984-b4aacf9e5e74"). InnerVolumeSpecName "kube-api-access-nzd9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:13:15.174347 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.174321 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f3124a-f8f6-403b-9984-b4aacf9e5e74-util" (OuterVolumeSpecName: "util") pod "99f3124a-f8f6-403b-9984-b4aacf9e5e74" (UID: "99f3124a-f8f6-403b-9984-b4aacf9e5e74"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:13:15.174421 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.174385 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e440e1a-c3c6-4903-88d5-fc70b40a4c17-util" (OuterVolumeSpecName: "util") pod "2e440e1a-c3c6-4903-88d5-fc70b40a4c17" (UID: "2e440e1a-c3c6-4903-88d5-fc70b40a4c17"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:13:15.268432 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.268345 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e440e1a-c3c6-4903-88d5-fc70b40a4c17-util\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:15.268432 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.268377 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7lvq\" (UniqueName: \"kubernetes.io/projected/2e440e1a-c3c6-4903-88d5-fc70b40a4c17-kube-api-access-c7lvq\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:15.268432 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.268389 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99f3124a-f8f6-403b-9984-b4aacf9e5e74-util\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:15.268432 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.268398 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e440e1a-c3c6-4903-88d5-fc70b40a4c17-bundle\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:15.268432 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.268405 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99f3124a-f8f6-403b-9984-b4aacf9e5e74-bundle\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:15.268432 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.268414 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzd9w\" (UniqueName: \"kubernetes.io/projected/99f3124a-f8f6-403b-9984-b4aacf9e5e74-kube-api-access-nzd9w\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:15.966113 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.966083 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx" Jul 13 17:13:15.966509 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.966077 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx" event={"ID":"99f3124a-f8f6-403b-9984-b4aacf9e5e74","Type":"ContainerDied","Data":"44d52161b09dcfaaa37c52f719dc49d2cc307dc62d076160bbb2d6b0fabf99f0"} Jul 13 17:13:15.966509 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.966222 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44d52161b09dcfaaa37c52f719dc49d2cc307dc62d076160bbb2d6b0fabf99f0" Jul 13 17:13:15.967974 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.967950 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2" Jul 13 17:13:15.968124 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.967972 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2" event={"ID":"2e440e1a-c3c6-4903-88d5-fc70b40a4c17","Type":"ContainerDied","Data":"c74cf9ab9a95b3ed228290bb3679cc4ab4f8021bb159f5e19eaf5a353510a966"} Jul 13 17:13:15.968124 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:15.968009 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c74cf9ab9a95b3ed228290bb3679cc4ab4f8021bb159f5e19eaf5a353510a966" Jul 13 17:13:16.042783 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:16.042739 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:16.042783 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:16.042791 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:16.048060 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:16.048032 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:16.101196 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:16.101174 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs" Jul 13 17:13:16.175862 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:16.175815 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d3aec72-80db-4422-a290-45527c4d9c6d-util\") pod \"9d3aec72-80db-4422-a290-45527c4d9c6d\" (UID: \"9d3aec72-80db-4422-a290-45527c4d9c6d\") " Jul 13 17:13:16.176031 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:16.175896 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfw8x\" (UniqueName: \"kubernetes.io/projected/9d3aec72-80db-4422-a290-45527c4d9c6d-kube-api-access-mfw8x\") pod \"9d3aec72-80db-4422-a290-45527c4d9c6d\" (UID: \"9d3aec72-80db-4422-a290-45527c4d9c6d\") " Jul 13 17:13:16.176031 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:16.175938 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d3aec72-80db-4422-a290-45527c4d9c6d-bundle\") pod \"9d3aec72-80db-4422-a290-45527c4d9c6d\" (UID: \"9d3aec72-80db-4422-a290-45527c4d9c6d\") " Jul 13 17:13:16.176519 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:16.176480 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d3aec72-80db-4422-a290-45527c4d9c6d-bundle" (OuterVolumeSpecName: "bundle") pod "9d3aec72-80db-4422-a290-45527c4d9c6d" (UID: "9d3aec72-80db-4422-a290-45527c4d9c6d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:13:16.178187 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:16.178164 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d3aec72-80db-4422-a290-45527c4d9c6d-kube-api-access-mfw8x" (OuterVolumeSpecName: "kube-api-access-mfw8x") pod "9d3aec72-80db-4422-a290-45527c4d9c6d" (UID: "9d3aec72-80db-4422-a290-45527c4d9c6d"). InnerVolumeSpecName "kube-api-access-mfw8x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:13:16.181616 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:16.181586 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d3aec72-80db-4422-a290-45527c4d9c6d-util" (OuterVolumeSpecName: "util") pod "9d3aec72-80db-4422-a290-45527c4d9c6d" (UID: "9d3aec72-80db-4422-a290-45527c4d9c6d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:13:16.276701 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:16.276606 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfw8x\" (UniqueName: \"kubernetes.io/projected/9d3aec72-80db-4422-a290-45527c4d9c6d-kube-api-access-mfw8x\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:16.276701 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:16.276645 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d3aec72-80db-4422-a290-45527c4d9c6d-bundle\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:16.276701 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:16.276654 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d3aec72-80db-4422-a290-45527c4d9c6d-util\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:16.980010 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:16.979982 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs" Jul 13 17:13:16.980358 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:16.980009 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs" event={"ID":"9d3aec72-80db-4422-a290-45527c4d9c6d","Type":"ContainerDied","Data":"cfe76f9955e24d2632778b07585f5f8dcaf2df87e1622065393aae6ad0e54d8a"} Jul 13 17:13:16.980358 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:16.980073 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cfe76f9955e24d2632778b07585f5f8dcaf2df87e1622065393aae6ad0e54d8a" Jul 13 17:13:16.984170 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:16.984147 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5978bd6596-pgchn" Jul 13 17:13:17.044312 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:17.044270 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5496c587bb-mq8nd"] Jul 13 17:13:17.294306 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:17.294219 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:17.384647 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:17.384606 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:17.442172 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:17.442139 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:23.820386 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:23.820353 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:23.917712 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:23.917674 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:24.223497 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:24.223466 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:25.557669 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:25.557635 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:25.617537 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:25.617500 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:29.002531 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:29.002499 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:32.602912 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:32.602880 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:37.859903 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.859865 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr"] Jul 13 17:13:37.860286 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860216 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e440e1a-c3c6-4903-88d5-fc70b40a4c17" containerName="pull" Jul 13 17:13:37.860286 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860228 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e440e1a-c3c6-4903-88d5-fc70b40a4c17" containerName="pull" Jul 13 17:13:37.860286 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860237 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99f3124a-f8f6-403b-9984-b4aacf9e5e74" containerName="extract" Jul 13 17:13:37.860286 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860242 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="99f3124a-f8f6-403b-9984-b4aacf9e5e74" containerName="extract" Jul 13 17:13:37.860286 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860252 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3236fc6b-d795-48f4-8298-2866758c75b5" containerName="pull" Jul 13 17:13:37.860286 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860257 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3236fc6b-d795-48f4-8298-2866758c75b5" containerName="pull" Jul 13 17:13:37.860286 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860270 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99f3124a-f8f6-403b-9984-b4aacf9e5e74" containerName="util" Jul 13 17:13:37.860286 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860279 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="99f3124a-f8f6-403b-9984-b4aacf9e5e74" containerName="util" Jul 13 17:13:37.860286 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860284 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d3aec72-80db-4422-a290-45527c4d9c6d" containerName="pull" Jul 13 17:13:37.860286 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860289 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d3aec72-80db-4422-a290-45527c4d9c6d" containerName="pull" Jul 13 17:13:37.860595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860296 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d3aec72-80db-4422-a290-45527c4d9c6d" containerName="extract" Jul 13 17:13:37.860595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860302 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d3aec72-80db-4422-a290-45527c4d9c6d" containerName="extract" Jul 13 17:13:37.860595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860308 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3236fc6b-d795-48f4-8298-2866758c75b5" containerName="util" Jul 13 17:13:37.860595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860313 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3236fc6b-d795-48f4-8298-2866758c75b5" containerName="util" Jul 13 17:13:37.860595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860318 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e440e1a-c3c6-4903-88d5-fc70b40a4c17" containerName="util" Jul 13 17:13:37.860595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860325 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e440e1a-c3c6-4903-88d5-fc70b40a4c17" containerName="util" Jul 13 17:13:37.860595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860330 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d3aec72-80db-4422-a290-45527c4d9c6d" containerName="util" Jul 13 17:13:37.860595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860334 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d3aec72-80db-4422-a290-45527c4d9c6d" containerName="util" Jul 13 17:13:37.860595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860347 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99f3124a-f8f6-403b-9984-b4aacf9e5e74" containerName="pull" Jul 13 17:13:37.860595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860355 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="99f3124a-f8f6-403b-9984-b4aacf9e5e74" containerName="pull" Jul 13 17:13:37.860595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860360 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3236fc6b-d795-48f4-8298-2866758c75b5" containerName="extract" Jul 13 17:13:37.860595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860364 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3236fc6b-d795-48f4-8298-2866758c75b5" containerName="extract" Jul 13 17:13:37.860595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860373 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e440e1a-c3c6-4903-88d5-fc70b40a4c17" containerName="extract" Jul 13 17:13:37.860595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860378 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e440e1a-c3c6-4903-88d5-fc70b40a4c17" containerName="extract" Jul 13 17:13:37.860595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860433 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d3aec72-80db-4422-a290-45527c4d9c6d" containerName="extract" Jul 13 17:13:37.860595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860441 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3236fc6b-d795-48f4-8298-2866758c75b5" containerName="extract" Jul 13 17:13:37.860595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860447 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e440e1a-c3c6-4903-88d5-fc70b40a4c17" containerName="extract" Jul 13 17:13:37.860595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.860453 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="99f3124a-f8f6-403b-9984-b4aacf9e5e74" containerName="extract" Jul 13 17:13:37.863595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.863572 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" Jul 13 17:13:37.867378 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.867355 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kbjnm\"" Jul 13 17:13:37.867530 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.867362 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 17:13:37.867782 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.867760 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 17:13:37.877278 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.877256 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr"] Jul 13 17:13:37.966102 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.966059 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jlsq\" (UniqueName: \"kubernetes.io/projected/4fc42769-e56a-4d20-9137-795e6830b7cc-kube-api-access-9jlsq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r87vr\" (UID: \"4fc42769-e56a-4d20-9137-795e6830b7cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" Jul 13 17:13:37.966102 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:37.966105 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fc42769-e56a-4d20-9137-795e6830b7cc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r87vr\" (UID: \"4fc42769-e56a-4d20-9137-795e6830b7cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" Jul 13 17:13:38.067071 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:38.067029 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jlsq\" (UniqueName: \"kubernetes.io/projected/4fc42769-e56a-4d20-9137-795e6830b7cc-kube-api-access-9jlsq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r87vr\" (UID: \"4fc42769-e56a-4d20-9137-795e6830b7cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" Jul 13 17:13:38.067283 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:38.067089 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fc42769-e56a-4d20-9137-795e6830b7cc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r87vr\" (UID: \"4fc42769-e56a-4d20-9137-795e6830b7cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" Jul 13 17:13:38.067529 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:38.067506 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fc42769-e56a-4d20-9137-795e6830b7cc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r87vr\" (UID: \"4fc42769-e56a-4d20-9137-795e6830b7cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" Jul 13 17:13:38.079150 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:38.079124 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jlsq\" (UniqueName: \"kubernetes.io/projected/4fc42769-e56a-4d20-9137-795e6830b7cc-kube-api-access-9jlsq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r87vr\" (UID: \"4fc42769-e56a-4d20-9137-795e6830b7cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" Jul 13 17:13:38.174343 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:38.174306 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" Jul 13 17:13:38.314953 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:38.314928 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr"] Jul 13 17:13:38.317235 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:13:38.317207 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fc42769_e56a_4d20_9137_795e6830b7cc.slice/crio-25e081c9e26553c1e28a5ed10bfea019811aa215c1b027b514a1c774b1011efa WatchSource:0}: Error finding container 25e081c9e26553c1e28a5ed10bfea019811aa215c1b027b514a1c774b1011efa: Status 404 returned error can't find the container with id 25e081c9e26553c1e28a5ed10bfea019811aa215c1b027b514a1c774b1011efa Jul 13 17:13:38.413632 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:38.413590 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:39.092265 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:39.092224 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" event={"ID":"4fc42769-e56a-4d20-9137-795e6830b7cc","Type":"ContainerStarted","Data":"25e081c9e26553c1e28a5ed10bfea019811aa215c1b027b514a1c774b1011efa"} Jul 13 17:13:39.205139 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:39.205105 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:41.649713 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:41.649680 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:42.066641 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.066549 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5496c587bb-mq8nd" podUID="8bac6b64-1ca5-494e-a65d-5507b3b071b2" containerName="console" containerID="cri-o://f0eafa474e9d4c387dbd94463fcde4af3230735f3033cd96cb19e981445be2c5" gracePeriod=15 Jul 13 17:13:42.323766 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.323694 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:13:42.406817 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.406785 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bac6b64-1ca5-494e-a65d-5507b3b071b2-console-serving-cert\") pod \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " Jul 13 17:13:42.407013 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.406831 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k27l9\" (UniqueName: \"kubernetes.io/projected/8bac6b64-1ca5-494e-a65d-5507b3b071b2-kube-api-access-k27l9\") pod \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " Jul 13 17:13:42.407013 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.406879 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-console-config\") pod \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " Jul 13 17:13:42.407132 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.407047 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bac6b64-1ca5-494e-a65d-5507b3b071b2-console-oauth-config\") pod \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " Jul 13 17:13:42.407190 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.407135 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-oauth-serving-cert\") pod \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " Jul 13 17:13:42.407190 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.407161 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-trusted-ca-bundle\") pod \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " Jul 13 17:13:42.407298 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.407193 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-service-ca\") pod \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\" (UID: \"8bac6b64-1ca5-494e-a65d-5507b3b071b2\") " Jul 13 17:13:42.407352 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.407290 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-console-config" (OuterVolumeSpecName: "console-config") pod "8bac6b64-1ca5-494e-a65d-5507b3b071b2" (UID: "8bac6b64-1ca5-494e-a65d-5507b3b071b2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:13:42.407655 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.407616 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-console-config\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:42.407876 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.407843 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-service-ca" (OuterVolumeSpecName: "service-ca") pod "8bac6b64-1ca5-494e-a65d-5507b3b071b2" (UID: "8bac6b64-1ca5-494e-a65d-5507b3b071b2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:13:42.407998 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.407875 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8bac6b64-1ca5-494e-a65d-5507b3b071b2" (UID: "8bac6b64-1ca5-494e-a65d-5507b3b071b2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:13:42.407998 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.407921 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8bac6b64-1ca5-494e-a65d-5507b3b071b2" (UID: "8bac6b64-1ca5-494e-a65d-5507b3b071b2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:13:42.409415 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.409389 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bac6b64-1ca5-494e-a65d-5507b3b071b2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8bac6b64-1ca5-494e-a65d-5507b3b071b2" (UID: "8bac6b64-1ca5-494e-a65d-5507b3b071b2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:13:42.409515 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.409490 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bac6b64-1ca5-494e-a65d-5507b3b071b2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8bac6b64-1ca5-494e-a65d-5507b3b071b2" (UID: "8bac6b64-1ca5-494e-a65d-5507b3b071b2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:13:42.409515 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.409505 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bac6b64-1ca5-494e-a65d-5507b3b071b2-kube-api-access-k27l9" (OuterVolumeSpecName: "kube-api-access-k27l9") pod "8bac6b64-1ca5-494e-a65d-5507b3b071b2" (UID: "8bac6b64-1ca5-494e-a65d-5507b3b071b2"). InnerVolumeSpecName "kube-api-access-k27l9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:13:42.508975 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.508937 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k27l9\" (UniqueName: \"kubernetes.io/projected/8bac6b64-1ca5-494e-a65d-5507b3b071b2-kube-api-access-k27l9\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:42.508975 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.508969 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bac6b64-1ca5-494e-a65d-5507b3b071b2-console-oauth-config\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:42.508975 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.508980 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-oauth-serving-cert\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:42.509209 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.508989 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-trusted-ca-bundle\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:42.509209 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.509007 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bac6b64-1ca5-494e-a65d-5507b3b071b2-service-ca\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:42.509209 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:42.509016 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bac6b64-1ca5-494e-a65d-5507b3b071b2-console-serving-cert\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:13:43.115567 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:43.115528 2564 generic.go:356] "Generic (PLEG): container finished" podID="8bac6b64-1ca5-494e-a65d-5507b3b071b2" containerID="f0eafa474e9d4c387dbd94463fcde4af3230735f3033cd96cb19e981445be2c5" exitCode=0 Jul 13 17:13:43.115567 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:43.115571 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5496c587bb-mq8nd" Jul 13 17:13:43.116172 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:43.115604 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5496c587bb-mq8nd" event={"ID":"8bac6b64-1ca5-494e-a65d-5507b3b071b2","Type":"ContainerDied","Data":"f0eafa474e9d4c387dbd94463fcde4af3230735f3033cd96cb19e981445be2c5"} Jul 13 17:13:43.116172 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:43.115668 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5496c587bb-mq8nd" event={"ID":"8bac6b64-1ca5-494e-a65d-5507b3b071b2","Type":"ContainerDied","Data":"32eed5036f73203e51026ccc5d378f366774101c3f047d2f165f29ca35326328"} Jul 13 17:13:43.116172 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:43.115689 2564 scope.go:117] "RemoveContainer" containerID="f0eafa474e9d4c387dbd94463fcde4af3230735f3033cd96cb19e981445be2c5" Jul 13 17:13:43.125165 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:43.125138 2564 scope.go:117] "RemoveContainer" containerID="f0eafa474e9d4c387dbd94463fcde4af3230735f3033cd96cb19e981445be2c5" Jul 13 17:13:43.125506 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:13:43.125484 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0eafa474e9d4c387dbd94463fcde4af3230735f3033cd96cb19e981445be2c5\": container with ID starting with f0eafa474e9d4c387dbd94463fcde4af3230735f3033cd96cb19e981445be2c5 not found: ID does not exist" containerID="f0eafa474e9d4c387dbd94463fcde4af3230735f3033cd96cb19e981445be2c5" Jul 13 17:13:43.125584 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:43.125515 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0eafa474e9d4c387dbd94463fcde4af3230735f3033cd96cb19e981445be2c5"} err="failed to get container status \"f0eafa474e9d4c387dbd94463fcde4af3230735f3033cd96cb19e981445be2c5\": rpc error: code = NotFound desc = could not find container \"f0eafa474e9d4c387dbd94463fcde4af3230735f3033cd96cb19e981445be2c5\": container with ID starting with f0eafa474e9d4c387dbd94463fcde4af3230735f3033cd96cb19e981445be2c5 not found: ID does not exist" Jul 13 17:13:43.145789 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:43.145723 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5496c587bb-mq8nd"] Jul 13 17:13:43.147643 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:43.147621 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5496c587bb-mq8nd"] Jul 13 17:13:43.446965 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:43.446923 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:44.426069 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:44.426035 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:44.535535 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:44.535496 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:44.854805 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:44.854702 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:44.960924 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:44.960883 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bac6b64-1ca5-494e-a65d-5507b3b071b2" path="/var/lib/kubelet/pods/8bac6b64-1ca5-494e-a65d-5507b3b071b2/volumes" Jul 13 17:13:46.486488 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:46.486448 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:47.139003 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:47.138961 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" event={"ID":"4fc42769-e56a-4d20-9137-795e6830b7cc","Type":"ContainerStarted","Data":"3d3b62905d19e0228d7c2dc11d4943622265bade7818bc2004ef4d1413fd08ce"} Jul 13 17:13:47.139178 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:47.139102 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" Jul 13 17:13:47.163878 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:47.163780 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" podStartSLOduration=1.583414334 podStartE2EDuration="10.163764389s" totalImagesPullingTime="8.580350055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:13:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:13:47.163387893 +0000 UTC m=+504.826163081" watchObservedRunningTime="2026-07-13 17:13:47.163764389 +0000 UTC m=+504.826539577" Jul 13 17:13:47.813344 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:47.813308 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:47.913636 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:47.913605 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:49.557705 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:49.557663 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:50.014560 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:50.014521 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:51.218610 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:51.218570 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:51.282235 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:51.282199 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:13:58.147185 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:58.147146 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" Jul 13 17:13:59.837837 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:59.837802 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr"] Jul 13 17:13:59.838261 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:59.837997 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" podUID="4fc42769-e56a-4d20-9137-795e6830b7cc" containerName="manager" containerID="cri-o://3d3b62905d19e0228d7c2dc11d4943622265bade7818bc2004ef4d1413fd08ce" gracePeriod=2 Jul 13 17:13:59.849324 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:59.849293 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr"] Jul 13 17:13:59.874653 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:59.874619 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w"] Jul 13 17:13:59.874977 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:59.874962 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fc42769-e56a-4d20-9137-795e6830b7cc" containerName="manager" Jul 13 17:13:59.875023 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:59.874979 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fc42769-e56a-4d20-9137-795e6830b7cc" containerName="manager" Jul 13 17:13:59.875023 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:59.874991 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bac6b64-1ca5-494e-a65d-5507b3b071b2" containerName="console" Jul 13 17:13:59.875023 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:59.874997 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bac6b64-1ca5-494e-a65d-5507b3b071b2" containerName="console" Jul 13 17:13:59.875114 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:59.875079 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fc42769-e56a-4d20-9137-795e6830b7cc" containerName="manager" Jul 13 17:13:59.875114 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:59.875087 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bac6b64-1ca5-494e-a65d-5507b3b071b2" containerName="console" Jul 13 17:13:59.877544 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:59.877515 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w" Jul 13 17:13:59.882779 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:13:59.882721 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-r87vr\" is forbidden: User \"system:node:ip-10-0-137-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-245.ec2.internal' and this object" podUID="4fc42769-e56a-4d20-9137-795e6830b7cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" Jul 13 17:13:59.905456 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:59.905423 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w"] Jul 13 17:13:59.964945 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:59.964915 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f9gk\" (UniqueName: \"kubernetes.io/projected/8e60256f-0f3c-4a3b-ace5-18c5a25b2770-kube-api-access-7f9gk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s7c7w\" (UID: \"8e60256f-0f3c-4a3b-ace5-18c5a25b2770\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w" Jul 13 17:13:59.965089 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:13:59.964966 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e60256f-0f3c-4a3b-ace5-18c5a25b2770-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s7c7w\" (UID: \"8e60256f-0f3c-4a3b-ace5-18c5a25b2770\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w" Jul 13 17:14:00.065489 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.065466 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e60256f-0f3c-4a3b-ace5-18c5a25b2770-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s7c7w\" (UID: \"8e60256f-0f3c-4a3b-ace5-18c5a25b2770\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w" Jul 13 17:14:00.065595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.065562 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f9gk\" (UniqueName: \"kubernetes.io/projected/8e60256f-0f3c-4a3b-ace5-18c5a25b2770-kube-api-access-7f9gk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s7c7w\" (UID: \"8e60256f-0f3c-4a3b-ace5-18c5a25b2770\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w" Jul 13 17:14:00.065879 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.065860 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e60256f-0f3c-4a3b-ace5-18c5a25b2770-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s7c7w\" (UID: \"8e60256f-0f3c-4a3b-ace5-18c5a25b2770\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w" Jul 13 17:14:00.074587 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.074557 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f9gk\" (UniqueName: \"kubernetes.io/projected/8e60256f-0f3c-4a3b-ace5-18c5a25b2770-kube-api-access-7f9gk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s7c7w\" (UID: \"8e60256f-0f3c-4a3b-ace5-18c5a25b2770\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w" Jul 13 17:14:00.090829 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.090764 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" Jul 13 17:14:00.093640 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:14:00.093616 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-r87vr\" is forbidden: User \"system:node:ip-10-0-137-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-245.ec2.internal' and this object" podUID="4fc42769-e56a-4d20-9137-795e6830b7cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" Jul 13 17:14:00.166779 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.166717 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jlsq\" (UniqueName: \"kubernetes.io/projected/4fc42769-e56a-4d20-9137-795e6830b7cc-kube-api-access-9jlsq\") pod \"4fc42769-e56a-4d20-9137-795e6830b7cc\" (UID: \"4fc42769-e56a-4d20-9137-795e6830b7cc\") " Jul 13 17:14:00.166920 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.166809 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fc42769-e56a-4d20-9137-795e6830b7cc-extensions-socket-volume\") pod \"4fc42769-e56a-4d20-9137-795e6830b7cc\" (UID: \"4fc42769-e56a-4d20-9137-795e6830b7cc\") " Jul 13 17:14:00.167308 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.167282 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fc42769-e56a-4d20-9137-795e6830b7cc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4fc42769-e56a-4d20-9137-795e6830b7cc" (UID: "4fc42769-e56a-4d20-9137-795e6830b7cc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:14:00.169034 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.169010 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fc42769-e56a-4d20-9137-795e6830b7cc-kube-api-access-9jlsq" (OuterVolumeSpecName: "kube-api-access-9jlsq") pod "4fc42769-e56a-4d20-9137-795e6830b7cc" (UID: "4fc42769-e56a-4d20-9137-795e6830b7cc"). InnerVolumeSpecName "kube-api-access-9jlsq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:14:00.203934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.203903 2564 generic.go:356] "Generic (PLEG): container finished" podID="4fc42769-e56a-4d20-9137-795e6830b7cc" containerID="3d3b62905d19e0228d7c2dc11d4943622265bade7818bc2004ef4d1413fd08ce" exitCode=0 Jul 13 17:14:00.204121 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.203950 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" Jul 13 17:14:00.204121 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.204004 2564 scope.go:117] "RemoveContainer" containerID="3d3b62905d19e0228d7c2dc11d4943622265bade7818bc2004ef4d1413fd08ce" Jul 13 17:14:00.213614 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:14:00.213582 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-r87vr\" is forbidden: User \"system:node:ip-10-0-137-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-245.ec2.internal' and this object" podUID="4fc42769-e56a-4d20-9137-795e6830b7cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" Jul 13 17:14:00.213708 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.213660 2564 scope.go:117] "RemoveContainer" containerID="3d3b62905d19e0228d7c2dc11d4943622265bade7818bc2004ef4d1413fd08ce" Jul 13 17:14:00.213978 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:14:00.213959 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d3b62905d19e0228d7c2dc11d4943622265bade7818bc2004ef4d1413fd08ce\": container with ID starting with 3d3b62905d19e0228d7c2dc11d4943622265bade7818bc2004ef4d1413fd08ce not found: ID does not exist" containerID="3d3b62905d19e0228d7c2dc11d4943622265bade7818bc2004ef4d1413fd08ce" Jul 13 17:14:00.214054 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.213989 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d3b62905d19e0228d7c2dc11d4943622265bade7818bc2004ef4d1413fd08ce"} err="failed to get container status \"3d3b62905d19e0228d7c2dc11d4943622265bade7818bc2004ef4d1413fd08ce\": rpc error: code = NotFound desc = could not find container \"3d3b62905d19e0228d7c2dc11d4943622265bade7818bc2004ef4d1413fd08ce\": container with ID starting with 3d3b62905d19e0228d7c2dc11d4943622265bade7818bc2004ef4d1413fd08ce not found: ID does not exist" Jul 13 17:14:00.218099 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:14:00.218075 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-r87vr\" is forbidden: User \"system:node:ip-10-0-137-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-245.ec2.internal' and this object" podUID="4fc42769-e56a-4d20-9137-795e6830b7cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r87vr" Jul 13 17:14:00.221237 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.221215 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w" Jul 13 17:14:00.268024 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.267992 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fc42769-e56a-4d20-9137-795e6830b7cc-extensions-socket-volume\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:14:00.268024 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.268021 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jlsq\" (UniqueName: \"kubernetes.io/projected/4fc42769-e56a-4d20-9137-795e6830b7cc-kube-api-access-9jlsq\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:14:00.363291 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.363168 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:00.386563 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.386538 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w"] Jul 13 17:14:00.388418 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:14:00.388394 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e60256f_0f3c_4a3b_ace5_18c5a25b2770.slice/crio-b0ecaf6331e54a5e3f71f601b791dbd3c213094f4d045c45f2d3bec3341cf493 WatchSource:0}: Error finding container b0ecaf6331e54a5e3f71f601b791dbd3c213094f4d045c45f2d3bec3341cf493: Status 404 returned error can't find the container with id b0ecaf6331e54a5e3f71f601b791dbd3c213094f4d045c45f2d3bec3341cf493 Jul 13 17:14:00.508482 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.508448 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:00.615046 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.614943 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:00.960319 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:00.960282 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fc42769-e56a-4d20-9137-795e6830b7cc" path="/var/lib/kubelet/pods/4fc42769-e56a-4d20-9137-795e6830b7cc/volumes" Jul 13 17:14:01.210849 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:01.210756 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w" event={"ID":"8e60256f-0f3c-4a3b-ace5-18c5a25b2770","Type":"ContainerStarted","Data":"a33e76bcd23a867a00d32e560ef932ebee23480673d50d35c78f750744bae972"} Jul 13 17:14:01.210849 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:01.210824 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w" Jul 13 17:14:01.210849 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:01.210847 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w" event={"ID":"8e60256f-0f3c-4a3b-ace5-18c5a25b2770","Type":"ContainerStarted","Data":"b0ecaf6331e54a5e3f71f601b791dbd3c213094f4d045c45f2d3bec3341cf493"} Jul 13 17:14:01.245886 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:01.245826 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w" podStartSLOduration=2.245810988 podStartE2EDuration="2.245810988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:13:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:14:01.244804031 +0000 UTC m=+518.907579218" watchObservedRunningTime="2026-07-13 17:14:01.245810988 +0000 UTC m=+518.908586176" Jul 13 17:14:02.753635 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:02.753596 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:02.858010 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:02.857947 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:05.580062 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:05.580007 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:05.681231 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:05.681200 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:09.137550 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:09.137512 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:09.236858 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:09.236816 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:12.218057 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:12.218030 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w" Jul 13 17:14:17.651717 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:17.651599 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:17.844518 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:17.844487 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:17.957103 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:17.956979 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:18.057243 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:18.057205 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:20.563146 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:20.563106 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:20.667849 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:20.667813 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:23.762165 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:23.762127 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:23.865296 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:23.865254 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:27.162306 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:27.162265 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:27.224442 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:27.224412 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:30.279435 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.279403 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w"] Jul 13 17:14:30.279972 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.279610 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w" podUID="8e60256f-0f3c-4a3b-ace5-18c5a25b2770" containerName="manager" containerID="cri-o://a33e76bcd23a867a00d32e560ef932ebee23480673d50d35c78f750744bae972" gracePeriod=10 Jul 13 17:14:30.530498 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.530436 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w" Jul 13 17:14:30.565331 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.565302 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql"] Jul 13 17:14:30.565638 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.565626 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e60256f-0f3c-4a3b-ace5-18c5a25b2770" containerName="manager" Jul 13 17:14:30.565687 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.565639 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e60256f-0f3c-4a3b-ace5-18c5a25b2770" containerName="manager" Jul 13 17:14:30.565722 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.565695 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e60256f-0f3c-4a3b-ace5-18c5a25b2770" containerName="manager" Jul 13 17:14:30.567694 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.567677 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql" Jul 13 17:14:30.602579 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.602550 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql"] Jul 13 17:14:30.633106 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.633076 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e60256f-0f3c-4a3b-ace5-18c5a25b2770-extensions-socket-volume\") pod \"8e60256f-0f3c-4a3b-ace5-18c5a25b2770\" (UID: \"8e60256f-0f3c-4a3b-ace5-18c5a25b2770\") " Jul 13 17:14:30.633279 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.633143 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7f9gk\" (UniqueName: \"kubernetes.io/projected/8e60256f-0f3c-4a3b-ace5-18c5a25b2770-kube-api-access-7f9gk\") pod \"8e60256f-0f3c-4a3b-ace5-18c5a25b2770\" (UID: \"8e60256f-0f3c-4a3b-ace5-18c5a25b2770\") " Jul 13 17:14:30.633449 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.633424 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e60256f-0f3c-4a3b-ace5-18c5a25b2770-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8e60256f-0f3c-4a3b-ace5-18c5a25b2770" (UID: "8e60256f-0f3c-4a3b-ace5-18c5a25b2770"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:14:30.635485 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.635461 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e60256f-0f3c-4a3b-ace5-18c5a25b2770-kube-api-access-7f9gk" (OuterVolumeSpecName: "kube-api-access-7f9gk") pod "8e60256f-0f3c-4a3b-ace5-18c5a25b2770" (UID: "8e60256f-0f3c-4a3b-ace5-18c5a25b2770"). InnerVolumeSpecName "kube-api-access-7f9gk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:14:30.734390 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.734349 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37a438ff-c325-4906-b03f-3ec7278eb5ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-z6vql\" (UID: \"37a438ff-c325-4906-b03f-3ec7278eb5ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql" Jul 13 17:14:30.734390 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.734394 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrzzm\" (UniqueName: \"kubernetes.io/projected/37a438ff-c325-4906-b03f-3ec7278eb5ef-kube-api-access-vrzzm\") pod \"kuadrant-operator-controller-manager-86878c769f-z6vql\" (UID: \"37a438ff-c325-4906-b03f-3ec7278eb5ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql" Jul 13 17:14:30.734655 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.734564 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f9gk\" (UniqueName: \"kubernetes.io/projected/8e60256f-0f3c-4a3b-ace5-18c5a25b2770-kube-api-access-7f9gk\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:14:30.734655 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.734591 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e60256f-0f3c-4a3b-ace5-18c5a25b2770-extensions-socket-volume\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:14:30.835115 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.835013 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37a438ff-c325-4906-b03f-3ec7278eb5ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-z6vql\" (UID: \"37a438ff-c325-4906-b03f-3ec7278eb5ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql" Jul 13 17:14:30.835115 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.835064 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrzzm\" (UniqueName: \"kubernetes.io/projected/37a438ff-c325-4906-b03f-3ec7278eb5ef-kube-api-access-vrzzm\") pod \"kuadrant-operator-controller-manager-86878c769f-z6vql\" (UID: \"37a438ff-c325-4906-b03f-3ec7278eb5ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql" Jul 13 17:14:30.835499 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.835476 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37a438ff-c325-4906-b03f-3ec7278eb5ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-z6vql\" (UID: \"37a438ff-c325-4906-b03f-3ec7278eb5ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql" Jul 13 17:14:30.845828 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.845805 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrzzm\" (UniqueName: \"kubernetes.io/projected/37a438ff-c325-4906-b03f-3ec7278eb5ef-kube-api-access-vrzzm\") pod \"kuadrant-operator-controller-manager-86878c769f-z6vql\" (UID: \"37a438ff-c325-4906-b03f-3ec7278eb5ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql" Jul 13 17:14:30.877836 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:30.877800 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql" Jul 13 17:14:31.035786 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:31.035754 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql"] Jul 13 17:14:31.037624 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:14:31.037597 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37a438ff_c325_4906_b03f_3ec7278eb5ef.slice/crio-6e1c3c45071922e1757c126c1e6dc4a9f94445d8566ec2183e4d81e05b4573c3 WatchSource:0}: Error finding container 6e1c3c45071922e1757c126c1e6dc4a9f94445d8566ec2183e4d81e05b4573c3: Status 404 returned error can't find the container with id 6e1c3c45071922e1757c126c1e6dc4a9f94445d8566ec2183e4d81e05b4573c3 Jul 13 17:14:31.361320 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:31.361204 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql" event={"ID":"37a438ff-c325-4906-b03f-3ec7278eb5ef","Type":"ContainerStarted","Data":"62c5310324d404192e03415795ee37d0e88fe34c22723109d10336508997b732"} Jul 13 17:14:31.361320 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:31.361274 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql" event={"ID":"37a438ff-c325-4906-b03f-3ec7278eb5ef","Type":"ContainerStarted","Data":"6e1c3c45071922e1757c126c1e6dc4a9f94445d8566ec2183e4d81e05b4573c3"} Jul 13 17:14:31.361320 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:31.361301 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql" Jul 13 17:14:31.362543 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:31.362511 2564 generic.go:356] "Generic (PLEG): container finished" podID="8e60256f-0f3c-4a3b-ace5-18c5a25b2770" containerID="a33e76bcd23a867a00d32e560ef932ebee23480673d50d35c78f750744bae972" exitCode=0 Jul 13 17:14:31.362662 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:31.362563 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w" Jul 13 17:14:31.362662 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:31.362592 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w" event={"ID":"8e60256f-0f3c-4a3b-ace5-18c5a25b2770","Type":"ContainerDied","Data":"a33e76bcd23a867a00d32e560ef932ebee23480673d50d35c78f750744bae972"} Jul 13 17:14:31.362662 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:31.362635 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w" event={"ID":"8e60256f-0f3c-4a3b-ace5-18c5a25b2770","Type":"ContainerDied","Data":"b0ecaf6331e54a5e3f71f601b791dbd3c213094f4d045c45f2d3bec3341cf493"} Jul 13 17:14:31.362662 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:31.362657 2564 scope.go:117] "RemoveContainer" containerID="a33e76bcd23a867a00d32e560ef932ebee23480673d50d35c78f750744bae972" Jul 13 17:14:31.371611 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:31.371584 2564 scope.go:117] "RemoveContainer" containerID="a33e76bcd23a867a00d32e560ef932ebee23480673d50d35c78f750744bae972" Jul 13 17:14:31.371929 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:14:31.371905 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a33e76bcd23a867a00d32e560ef932ebee23480673d50d35c78f750744bae972\": container with ID starting with a33e76bcd23a867a00d32e560ef932ebee23480673d50d35c78f750744bae972 not found: ID does not exist" containerID="a33e76bcd23a867a00d32e560ef932ebee23480673d50d35c78f750744bae972" Jul 13 17:14:31.372021 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:31.371941 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a33e76bcd23a867a00d32e560ef932ebee23480673d50d35c78f750744bae972"} err="failed to get container status \"a33e76bcd23a867a00d32e560ef932ebee23480673d50d35c78f750744bae972\": rpc error: code = NotFound desc = could not find container \"a33e76bcd23a867a00d32e560ef932ebee23480673d50d35c78f750744bae972\": container with ID starting with a33e76bcd23a867a00d32e560ef932ebee23480673d50d35c78f750744bae972 not found: ID does not exist" Jul 13 17:14:31.381326 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:31.381278 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql" podStartSLOduration=1.381264684 podStartE2EDuration="1.381264684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:14:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:14:31.380593885 +0000 UTC m=+549.043369099" watchObservedRunningTime="2026-07-13 17:14:31.381264684 +0000 UTC m=+549.044039871" Jul 13 17:14:31.399144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:31.399108 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w"] Jul 13 17:14:31.403265 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:31.403237 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7c7w"] Jul 13 17:14:32.960243 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:32.960212 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e60256f-0f3c-4a3b-ace5-18c5a25b2770" path="/var/lib/kubelet/pods/8e60256f-0f3c-4a3b-ace5-18c5a25b2770/volumes" Jul 13 17:14:35.883055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:35.883021 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:35.982898 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:35.982854 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:36.773575 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:36.773526 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:36.839879 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:36.839843 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:42.371601 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:42.371568 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql" Jul 13 17:14:46.718459 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.718415 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb"] Jul 13 17:14:46.721807 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.721782 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.726064 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.726040 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-mt7t6\"" Jul 13 17:14:46.737528 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.737501 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb"] Jul 13 17:14:46.760945 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.760916 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:46.866646 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.866606 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.866646 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.866648 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.866901 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.866674 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.866901 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.866703 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.866901 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.866754 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.866901 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.866779 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmf2q\" (UniqueName: \"kubernetes.io/projected/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-kube-api-access-mmf2q\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.866901 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.866812 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.866901 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.866829 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.866901 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.866844 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.967439 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.967383 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.967439 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.967448 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.967439 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.967489 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.967842 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.967512 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.967842 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.967534 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.967842 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.967666 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmf2q\" (UniqueName: \"kubernetes.io/projected/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-kube-api-access-mmf2q\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.967842 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.967750 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.967842 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.967782 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.967842 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.967809 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.968160 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.967956 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.968160 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.968110 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.968160 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.968136 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.968313 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.968189 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.969008 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.968929 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.970276 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.970254 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.970367 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.970326 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.977531 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.977511 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:46.979699 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:46.979676 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmf2q\" (UniqueName: \"kubernetes.io/projected/6db06f8c-f4ec-49d9-94c1-9dade34c33f8-kube-api-access-mmf2q\") pod \"maas-default-gateway-openshift-default-56654dcf7-84jgb\" (UID: \"6db06f8c-f4ec-49d9-94c1-9dade34c33f8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:47.032405 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:47.032368 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:47.177105 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:47.177073 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb"] Jul 13 17:14:47.177999 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:14:47.177967 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6db06f8c_f4ec_49d9_94c1_9dade34c33f8.slice/crio-638ec76d3ae0fdd8b64faf92041fc4355c9682966c200f923ac6acec3abb63d7 WatchSource:0}: Error finding container 638ec76d3ae0fdd8b64faf92041fc4355c9682966c200f923ac6acec3abb63d7: Status 404 returned error can't find the container with id 638ec76d3ae0fdd8b64faf92041fc4355c9682966c200f923ac6acec3abb63d7 Jul 13 17:14:47.179985 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:47.179954 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:14:47.180094 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:47.180041 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:14:47.180161 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:47.180098 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:14:47.191870 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:47.191851 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:47.443823 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:47.443780 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" event={"ID":"6db06f8c-f4ec-49d9-94c1-9dade34c33f8","Type":"ContainerStarted","Data":"4484ff10081f3692ca6d66680a8dc1551728821c774efd342af6f0e4107343d3"} Jul 13 17:14:47.444006 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:47.443852 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" event={"ID":"6db06f8c-f4ec-49d9-94c1-9dade34c33f8","Type":"ContainerStarted","Data":"638ec76d3ae0fdd8b64faf92041fc4355c9682966c200f923ac6acec3abb63d7"} Jul 13 17:14:47.498167 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:47.498069 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" podStartSLOduration=1.498045888 podStartE2EDuration="1.498045888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:14:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:14:47.497286685 +0000 UTC m=+565.160061864" watchObservedRunningTime="2026-07-13 17:14:47.498045888 +0000 UTC m=+565.160821076" Jul 13 17:14:48.033141 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:48.033105 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:48.037662 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:48.037636 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:48.448815 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:48.448784 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:48.449785 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:48.449768 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:14:49.312958 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:49.312924 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:49.433025 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:49.432982 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:49.532280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:49.532251 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:49.688639 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:49.688607 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:14:49.691900 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:49.691882 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-9bmg9" Jul 13 17:14:49.692792 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:49.692767 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/709376d2-4a59-46e5-a1a1-987b0a525e95-config-file\") pod \"limitador-limitador-54d6d988d5-9bmg9\" (UID: \"709376d2-4a59-46e5-a1a1-987b0a525e95\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9bmg9" Jul 13 17:14:49.692880 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:49.692831 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rttlh\" (UniqueName: \"kubernetes.io/projected/709376d2-4a59-46e5-a1a1-987b0a525e95-kube-api-access-rttlh\") pod \"limitador-limitador-54d6d988d5-9bmg9\" (UID: \"709376d2-4a59-46e5-a1a1-987b0a525e95\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9bmg9" Jul 13 17:14:49.694843 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:49.694818 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 13 17:14:49.694954 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:49.694820 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hzghr\"" Jul 13 17:14:49.700272 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:49.700249 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:14:49.733562 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:49.733522 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:14:49.793859 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:49.793827 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rttlh\" (UniqueName: \"kubernetes.io/projected/709376d2-4a59-46e5-a1a1-987b0a525e95-kube-api-access-rttlh\") pod \"limitador-limitador-54d6d988d5-9bmg9\" (UID: \"709376d2-4a59-46e5-a1a1-987b0a525e95\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9bmg9" Jul 13 17:14:49.794033 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:49.793908 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/709376d2-4a59-46e5-a1a1-987b0a525e95-config-file\") pod \"limitador-limitador-54d6d988d5-9bmg9\" (UID: \"709376d2-4a59-46e5-a1a1-987b0a525e95\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9bmg9" Jul 13 17:14:49.794612 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:49.794590 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/709376d2-4a59-46e5-a1a1-987b0a525e95-config-file\") pod \"limitador-limitador-54d6d988d5-9bmg9\" (UID: \"709376d2-4a59-46e5-a1a1-987b0a525e95\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9bmg9" Jul 13 17:14:49.802636 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:49.802606 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rttlh\" (UniqueName: \"kubernetes.io/projected/709376d2-4a59-46e5-a1a1-987b0a525e95-kube-api-access-rttlh\") pod \"limitador-limitador-54d6d988d5-9bmg9\" (UID: \"709376d2-4a59-46e5-a1a1-987b0a525e95\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9bmg9" Jul 13 17:14:50.003980 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:50.003876 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-9bmg9" Jul 13 17:14:50.157160 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:50.157112 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:50.158940 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:50.158913 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:14:50.160283 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:14:50.160257 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod709376d2_4a59_46e5_a1a1_987b0a525e95.slice/crio-d0689e1d68405c2e0a0c0f7628fb40fe63b7f5cad14d77325c0e17668dc892fc WatchSource:0}: Error finding container d0689e1d68405c2e0a0c0f7628fb40fe63b7f5cad14d77325c0e17668dc892fc: Status 404 returned error can't find the container with id d0689e1d68405c2e0a0c0f7628fb40fe63b7f5cad14d77325c0e17668dc892fc Jul 13 17:14:50.219525 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:50.219486 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:50.461056 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:50.461016 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-9bmg9" event={"ID":"709376d2-4a59-46e5-a1a1-987b0a525e95","Type":"ContainerStarted","Data":"d0689e1d68405c2e0a0c0f7628fb40fe63b7f5cad14d77325c0e17668dc892fc"} Jul 13 17:14:53.480107 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:53.480072 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-9bmg9" event={"ID":"709376d2-4a59-46e5-a1a1-987b0a525e95","Type":"ContainerStarted","Data":"6591b63dce59ff71587aa58b937fd702e3fdd3eb0af86aa9b7fbd36954ccdf1f"} Jul 13 17:14:53.480518 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:53.480203 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-9bmg9" Jul 13 17:14:53.510880 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:53.506515 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-9bmg9" podStartSLOduration=1.792958305 podStartE2EDuration="4.506493698s" totalImagesPullingTime="2.713535393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:14:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:14:53.50527027 +0000 UTC m=+571.168045459" watchObservedRunningTime="2026-07-13 17:14:53.506493698 +0000 UTC m=+571.169268887" Jul 13 17:14:55.055580 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:55.055545 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:14:55.111578 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:14:55.111544 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:15:04.486506 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:15:04.486474 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-9bmg9" Jul 13 17:15:09.927627 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:15:09.927589 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:15:10.010564 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:15:10.010526 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:15:10.409551 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:15:10.409518 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:15:22.900317 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:15:22.900285 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:15:23.478019 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:15:23.477984 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:15:29.039420 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:15:29.039380 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:15:30.109721 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:15:30.109688 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:15:32.598000 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:15:32.597962 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:15:32.730135 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:15:32.730101 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:15:53.130315 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:15:53.130279 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:15:53.189738 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:15:53.189701 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:16:06.317331 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:06.317294 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:16:06.414321 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:06.414292 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:16:06.535692 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:06.535641 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:16:06.793515 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:06.793448 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:16:07.198095 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:07.198052 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:16:07.303001 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:07.302964 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:16:07.387068 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:07.387029 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:16:14.951969 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:14.951936 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:16:15.897637 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:15.897596 2564 generic.go:356] "Generic (PLEG): container finished" podID="6db06f8c-f4ec-49d9-94c1-9dade34c33f8" containerID="4484ff10081f3692ca6d66680a8dc1551728821c774efd342af6f0e4107343d3" exitCode=0 Jul 13 17:16:15.897864 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:15.897672 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" event={"ID":"6db06f8c-f4ec-49d9-94c1-9dade34c33f8","Type":"ContainerDied","Data":"4484ff10081f3692ca6d66680a8dc1551728821c774efd342af6f0e4107343d3"} Jul 13 17:16:15.898501 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:15.898466 2564 scope.go:117] "RemoveContainer" containerID="4484ff10081f3692ca6d66680a8dc1551728821c774efd342af6f0e4107343d3" Jul 13 17:16:15.899634 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:15.899605 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:16:15.899766 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:15.899673 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:16:15.899766 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:15.899709 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:16:15.931077 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:15.931045 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:16:16.033573 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:16.033528 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:16:16.904979 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:16.904938 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" event={"ID":"6db06f8c-f4ec-49d9-94c1-9dade34c33f8","Type":"ContainerStarted","Data":"b23f1ab7eadff9455495ee3bd9d7481880d91d2685886f87dfae3fe48dfbdd7f"} Jul 13 17:16:17.033288 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:17.033252 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:16:17.034412 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:17.034379 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-84jgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 13 17:16:17.034842 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:17.034439 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" podUID="6db06f8c-f4ec-49d9-94c1-9dade34c33f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 13 17:16:18.033076 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:18.033041 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-84jgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 13 17:16:18.033274 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:18.033098 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" podUID="6db06f8c-f4ec-49d9-94c1-9dade34c33f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 13 17:16:18.449339 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:18.449303 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:16:19.033190 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:19.033152 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-84jgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 13 17:16:19.033373 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:19.033209 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" podUID="6db06f8c-f4ec-49d9-94c1-9dade34c33f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 13 17:16:20.032853 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:20.032814 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-84jgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 13 17:16:20.033302 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:20.032881 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" podUID="6db06f8c-f4ec-49d9-94c1-9dade34c33f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 13 17:16:21.037694 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:21.037661 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:16:21.038706 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:21.038684 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-84jgb" Jul 13 17:16:22.371558 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:22.371525 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:16:22.742572 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:22.742536 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:16:46.553795 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:46.553714 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:16:47.327426 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:16:47.327389 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:17:05.533823 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:17:05.533790 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:17:06.729520 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:17:06.729482 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:17:22.656513 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:17:22.656430 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:17:24.427575 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:17:24.427529 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:17:30.527368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:17:30.527335 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:17:32.629120 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:17:32.629083 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:17:35.738663 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:17:35.738628 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:17:44.231703 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:17:44.231664 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:18:35.005553 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:18:35.005517 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:18:35.105443 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:18:35.105411 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:18:36.238331 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:18:36.238298 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:18:36.345480 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:18:36.345446 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:19:26.968878 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:19:26.968844 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:19:27.026010 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:19:27.025975 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:19:38.234996 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:19:38.234950 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:19:47.919119 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:19:47.919078 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:19:58.528017 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:19:58.527974 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:20:07.036022 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:20:07.035981 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:20:18.233947 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:20:18.233862 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:20:20.584606 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:20:20.584575 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:20:22.024604 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:20:22.024560 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:20:45.127794 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:20:45.127758 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:20:54.837470 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:20:54.837431 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:21:00.260425 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:21:00.260394 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:21:03.910770 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:21:03.910687 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:21:04.036356 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:21:04.036323 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:21:04.191222 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:21:04.191137 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:21:09.916830 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:21:09.916782 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:21:10.022033 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:21:10.021997 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:21:10.167815 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:21:10.167687 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:21:10.613311 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:21:10.613217 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:21:16.229321 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:21:16.229271 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:21:16.325910 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:21:16.325875 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:21:33.751244 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:21:33.751200 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:21:43.625708 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:21:43.625626 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:21:54.331274 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:21:54.331231 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:22:01.830421 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:22:01.830388 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:22:13.148141 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:22:13.148105 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:22:19.765672 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:22:19.765636 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:22:22.932306 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:22:22.932259 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:22:34.638999 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:22:34.638960 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:22:44.845647 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:22:44.845599 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:22:57.958976 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:22:57.958937 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:23:06.146080 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:23:06.146045 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:23:18.027932 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:23:18.027842 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:23:43.623771 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:23:43.623718 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:23:50.914607 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:23:50.914570 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:23:50.964295 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:23:50.964263 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:23:53.835624 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:23:53.835593 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:24:09.025002 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:24:09.024964 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:24:20.530438 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:24:20.530402 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:24:30.529035 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:24:30.528993 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:24:43.022214 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:24:43.022176 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:25:14.090859 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:25:14.090825 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql"] Jul 13 17:25:14.091426 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:25:14.091019 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql" podUID="37a438ff-c325-4906-b03f-3ec7278eb5ef" containerName="manager" containerID="cri-o://62c5310324d404192e03415795ee37d0e88fe34c22723109d10336508997b732" gracePeriod=10 Jul 13 17:25:14.372771 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:25:14.372652 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:25:17.648679 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:25:17.648645 2564 generic.go:356] "Generic (PLEG): container finished" podID="37a438ff-c325-4906-b03f-3ec7278eb5ef" containerID="62c5310324d404192e03415795ee37d0e88fe34c22723109d10336508997b732" exitCode=0 Jul 13 17:25:17.649089 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:25:17.648747 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql" event={"ID":"37a438ff-c325-4906-b03f-3ec7278eb5ef","Type":"ContainerDied","Data":"62c5310324d404192e03415795ee37d0e88fe34c22723109d10336508997b732"} Jul 13 17:25:17.750551 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:25:17.750523 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql" Jul 13 17:25:17.907671 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:25:17.907639 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37a438ff-c325-4906-b03f-3ec7278eb5ef-extensions-socket-volume\") pod \"37a438ff-c325-4906-b03f-3ec7278eb5ef\" (UID: \"37a438ff-c325-4906-b03f-3ec7278eb5ef\") " Jul 13 17:25:17.907857 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:25:17.907684 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrzzm\" (UniqueName: \"kubernetes.io/projected/37a438ff-c325-4906-b03f-3ec7278eb5ef-kube-api-access-vrzzm\") pod \"37a438ff-c325-4906-b03f-3ec7278eb5ef\" (UID: \"37a438ff-c325-4906-b03f-3ec7278eb5ef\") " Jul 13 17:25:17.908118 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:25:17.908085 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37a438ff-c325-4906-b03f-3ec7278eb5ef-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "37a438ff-c325-4906-b03f-3ec7278eb5ef" (UID: "37a438ff-c325-4906-b03f-3ec7278eb5ef"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:25:17.909950 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:25:17.909927 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37a438ff-c325-4906-b03f-3ec7278eb5ef-kube-api-access-vrzzm" (OuterVolumeSpecName: "kube-api-access-vrzzm") pod "37a438ff-c325-4906-b03f-3ec7278eb5ef" (UID: "37a438ff-c325-4906-b03f-3ec7278eb5ef"). InnerVolumeSpecName "kube-api-access-vrzzm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:25:18.009372 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:25:18.009325 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37a438ff-c325-4906-b03f-3ec7278eb5ef-extensions-socket-volume\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:25:18.009372 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:25:18.009362 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrzzm\" (UniqueName: \"kubernetes.io/projected/37a438ff-c325-4906-b03f-3ec7278eb5ef-kube-api-access-vrzzm\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:25:18.655576 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:25:18.655543 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql" Jul 13 17:25:18.656071 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:25:18.655581 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql" event={"ID":"37a438ff-c325-4906-b03f-3ec7278eb5ef","Type":"ContainerDied","Data":"6e1c3c45071922e1757c126c1e6dc4a9f94445d8566ec2183e4d81e05b4573c3"} Jul 13 17:25:18.656071 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:25:18.655644 2564 scope.go:117] "RemoveContainer" containerID="62c5310324d404192e03415795ee37d0e88fe34c22723109d10336508997b732" Jul 13 17:25:18.678715 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:25:18.678664 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql"] Jul 13 17:25:18.681564 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:25:18.681537 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z6vql"] Jul 13 17:25:18.961122 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:25:18.961008 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37a438ff-c325-4906-b03f-3ec7278eb5ef" path="/var/lib/kubelet/pods/37a438ff-c325-4906-b03f-3ec7278eb5ef/volumes" Jul 13 17:26:15.240240 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.240203 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mcm8"] Jul 13 17:26:15.240789 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.240540 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37a438ff-c325-4906-b03f-3ec7278eb5ef" containerName="manager" Jul 13 17:26:15.240789 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.240552 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="37a438ff-c325-4906-b03f-3ec7278eb5ef" containerName="manager" Jul 13 17:26:15.240789 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.240610 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="37a438ff-c325-4906-b03f-3ec7278eb5ef" containerName="manager" Jul 13 17:26:15.242592 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.242575 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mcm8" Jul 13 17:26:15.245641 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.245618 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kbjnm\"" Jul 13 17:26:15.256457 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.256433 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mcm8"] Jul 13 17:26:15.313783 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.313715 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-css5w\" (UniqueName: \"kubernetes.io/projected/ee31e5fa-ca47-4680-a3b4-a7c517ad0506-kube-api-access-css5w\") pod \"kuadrant-operator-controller-manager-86878c769f-6mcm8\" (UID: \"ee31e5fa-ca47-4680-a3b4-a7c517ad0506\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mcm8" Jul 13 17:26:15.313959 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.313881 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee31e5fa-ca47-4680-a3b4-a7c517ad0506-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6mcm8\" (UID: \"ee31e5fa-ca47-4680-a3b4-a7c517ad0506\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mcm8" Jul 13 17:26:15.414893 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.414850 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee31e5fa-ca47-4680-a3b4-a7c517ad0506-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6mcm8\" (UID: \"ee31e5fa-ca47-4680-a3b4-a7c517ad0506\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mcm8" Jul 13 17:26:15.415088 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.414904 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-css5w\" (UniqueName: \"kubernetes.io/projected/ee31e5fa-ca47-4680-a3b4-a7c517ad0506-kube-api-access-css5w\") pod \"kuadrant-operator-controller-manager-86878c769f-6mcm8\" (UID: \"ee31e5fa-ca47-4680-a3b4-a7c517ad0506\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mcm8" Jul 13 17:26:15.415229 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.415205 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee31e5fa-ca47-4680-a3b4-a7c517ad0506-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6mcm8\" (UID: \"ee31e5fa-ca47-4680-a3b4-a7c517ad0506\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mcm8" Jul 13 17:26:15.424372 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.424346 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-css5w\" (UniqueName: \"kubernetes.io/projected/ee31e5fa-ca47-4680-a3b4-a7c517ad0506-kube-api-access-css5w\") pod \"kuadrant-operator-controller-manager-86878c769f-6mcm8\" (UID: \"ee31e5fa-ca47-4680-a3b4-a7c517ad0506\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mcm8" Jul 13 17:26:15.553150 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.553038 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mcm8" Jul 13 17:26:15.704094 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.704070 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mcm8"] Jul 13 17:26:15.706377 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:26:15.706349 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee31e5fa_ca47_4680_a3b4_a7c517ad0506.slice/crio-ee79e27558a0b35d742d7cf75f6808ca71adce669c90aed32cad9c85a2fe5ac0 WatchSource:0}: Error finding container ee79e27558a0b35d742d7cf75f6808ca71adce669c90aed32cad9c85a2fe5ac0: Status 404 returned error can't find the container with id ee79e27558a0b35d742d7cf75f6808ca71adce669c90aed32cad9c85a2fe5ac0 Jul 13 17:26:15.952928 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.952888 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mcm8" event={"ID":"ee31e5fa-ca47-4680-a3b4-a7c517ad0506","Type":"ContainerStarted","Data":"788ac1ab30230acbb5cbc4e39265af75ee1b3a0468e0fd56d50ce8b91976cc80"} Jul 13 17:26:15.953137 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.952966 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mcm8" event={"ID":"ee31e5fa-ca47-4680-a3b4-a7c517ad0506","Type":"ContainerStarted","Data":"ee79e27558a0b35d742d7cf75f6808ca71adce669c90aed32cad9c85a2fe5ac0"} Jul 13 17:26:15.953137 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.953024 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mcm8" Jul 13 17:26:15.974765 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:15.974693 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mcm8" podStartSLOduration=0.974677959 podStartE2EDuration="974.677959ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:26:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:26:15.973274161 +0000 UTC m=+1253.636049343" watchObservedRunningTime="2026-07-13 17:26:15.974677959 +0000 UTC m=+1253.637453146" Jul 13 17:26:26.961245 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:26.961212 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mcm8" Jul 13 17:26:36.131079 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:36.131037 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:26:45.338937 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:45.338897 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:26:57.928385 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:26:57.928344 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:27:09.935942 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:27:09.935906 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:27:19.726240 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:27:19.726202 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:27:31.431060 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:27:31.431020 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:27:40.029674 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:27:40.029638 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:27:50.924406 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:27:50.924367 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:28:01.233574 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:28:01.233538 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:28:08.047109 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:28:08.047075 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:28:09.355553 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:28:09.355512 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:28:10.341490 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:28:10.341455 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:28:21.636796 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:28:21.636758 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:28:43.236199 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:28:43.236161 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:28:52.824491 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:28:52.824446 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:29:02.823074 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:29:02.823039 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:29:12.729121 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:29:12.729088 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:29:19.727001 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:29:19.726907 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:29:34.839713 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:29:34.839665 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:29:49.231337 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:29:49.231302 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:29:58.833337 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:29:58.833300 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:30:00.140489 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:00.140448 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732730-m66p9"] Jul 13 17:30:00.142998 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:00.142981 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732730-m66p9" Jul 13 17:30:00.146143 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:00.146116 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k25z5\"" Jul 13 17:30:00.152210 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:00.152191 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732730-m66p9" Jul 13 17:30:00.161040 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:00.161018 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732730-m66p9"] Jul 13 17:30:00.290762 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:00.290717 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732730-m66p9"] Jul 13 17:30:00.292656 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:30:00.292617 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ecb4eb4_0272_46e1_8020_fc2c220122e8.slice/crio-df6b285ebff656d68e7ba1df9109f5991644f177bc3420225b6f40ad4ae48eb4 WatchSource:0}: Error finding container df6b285ebff656d68e7ba1df9109f5991644f177bc3420225b6f40ad4ae48eb4: Status 404 returned error can't find the container with id df6b285ebff656d68e7ba1df9109f5991644f177bc3420225b6f40ad4ae48eb4 Jul 13 17:30:00.294605 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:00.294582 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 17:30:01.108541 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:01.108506 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732730-m66p9" event={"ID":"2ecb4eb4-0272-46e1-8020-fc2c220122e8","Type":"ContainerStarted","Data":"df6b285ebff656d68e7ba1df9109f5991644f177bc3420225b6f40ad4ae48eb4"} Jul 13 17:30:05.132821 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:05.132785 2564 generic.go:356] "Generic (PLEG): container finished" podID="2ecb4eb4-0272-46e1-8020-fc2c220122e8" containerID="e7a1f9bc35a2c6e1b971e7f92bd28a61b11fa3cc951d3657632aaf64a6ca24f0" exitCode=0 Jul 13 17:30:05.133246 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:05.132881 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732730-m66p9" event={"ID":"2ecb4eb4-0272-46e1-8020-fc2c220122e8","Type":"ContainerDied","Data":"e7a1f9bc35a2c6e1b971e7f92bd28a61b11fa3cc951d3657632aaf64a6ca24f0"} Jul 13 17:30:06.264866 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:06.264840 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732730-m66p9" Jul 13 17:30:07.143749 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:07.143696 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732730-m66p9" event={"ID":"2ecb4eb4-0272-46e1-8020-fc2c220122e8","Type":"ContainerDied","Data":"df6b285ebff656d68e7ba1df9109f5991644f177bc3420225b6f40ad4ae48eb4"} Jul 13 17:30:07.143749 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:07.143723 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732730-m66p9" Jul 13 17:30:07.143969 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:07.143774 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df6b285ebff656d68e7ba1df9109f5991644f177bc3420225b6f40ad4ae48eb4" Jul 13 17:30:12.630693 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:12.630654 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:30:22.132406 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:22.132361 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:30:35.723776 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:35.723718 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:30:45.127935 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:45.127895 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:30:52.070591 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:52.070546 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:30:55.040598 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:55.040552 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:30:57.611430 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:30:57.611390 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:31:07.138082 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:31:07.138040 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:31:18.826566 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:31:18.826524 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:31:28.036502 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:31:28.036460 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:31:38.670658 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:31:38.670607 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:31:41.324685 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:31:41.324647 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:31:52.937635 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:31:52.937600 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:32:03.833544 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:03.833504 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:32:05.613092 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:05.613046 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:32:12.819030 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:12.818988 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:32:15.028596 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:15.028506 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:32:21.080985 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:21.080936 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:32:26.172007 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:26.171960 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:32:26.527138 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:26.527045 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:32:28.987576 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:28.987538 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:32:29.087969 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:29.087941 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:32:29.421972 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:29.421939 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:32:35.213724 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.213689 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:32:35.268693 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.268653 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:32:35.395390 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.395353 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx"] Jul 13 17:32:35.395763 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.395749 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ecb4eb4-0272-46e1-8020-fc2c220122e8" containerName="cleanup" Jul 13 17:32:35.395836 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.395766 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ecb4eb4-0272-46e1-8020-fc2c220122e8" containerName="cleanup" Jul 13 17:32:35.395877 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.395836 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ecb4eb4-0272-46e1-8020-fc2c220122e8" containerName="cleanup" Jul 13 17:32:35.398144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.398121 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.402177 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.402157 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-85580aa7-openshift-default-dockercfg-tvz59\"" Jul 13 17:32:35.428338 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.428305 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:32:35.429516 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.429489 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx"] Jul 13 17:32:35.490588 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.490488 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr9v6\" (UniqueName: \"kubernetes.io/projected/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-kube-api-access-wr9v6\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.490588 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.490544 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-envoy\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.490837 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.490641 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-token\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.490837 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.490673 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-data\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.490837 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.490698 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istiod-ca-cert\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.490837 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.490767 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-credential-socket\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.490837 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.490788 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-workload-socket\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.490837 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.490812 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-workload-certs\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.491050 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.490850 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-podinfo\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.528201 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.528166 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:32:35.591522 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.591487 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-token\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.591682 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.591526 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-data\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.591682 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.591560 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istiod-ca-cert\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.591682 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.591597 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-credential-socket\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.591682 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.591628 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-workload-socket\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.591682 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.591667 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-workload-certs\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.592007 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.591705 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-podinfo\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.592007 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.591794 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr9v6\" (UniqueName: \"kubernetes.io/projected/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-kube-api-access-wr9v6\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.592007 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.591945 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-envoy\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.592164 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.592018 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-credential-socket\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.592164 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.592066 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-data\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.592164 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.592121 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-workload-socket\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.592280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.592159 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-workload-certs\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.592370 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.592348 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istiod-ca-cert\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.594382 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.594365 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-envoy\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.594533 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.594515 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-podinfo\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.601860 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.601837 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-token\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.602349 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.602326 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr9v6\" (UniqueName: \"kubernetes.io/projected/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-kube-api-access-wr9v6\") pod \"e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.708634 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.708594 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:35.846928 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.846899 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx"] Jul 13 17:32:35.848534 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:32:35.848504 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bf45b3e_4198_45a9_8040_7a62ce3d07d1.slice/crio-611c0eea6a56502d5814ea0ad265827b01b7019893a16efa99aea2fcbc3d8975 WatchSource:0}: Error finding container 611c0eea6a56502d5814ea0ad265827b01b7019893a16efa99aea2fcbc3d8975: Status 404 returned error can't find the container with id 611c0eea6a56502d5814ea0ad265827b01b7019893a16efa99aea2fcbc3d8975 Jul 13 17:32:35.850678 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.850646 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:32:35.850774 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.850709 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:32:35.850774 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.850756 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:32:35.903686 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:35.903595 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" event={"ID":"6bf45b3e-4198-45a9-8040-7a62ce3d07d1","Type":"ContainerStarted","Data":"611c0eea6a56502d5814ea0ad265827b01b7019893a16efa99aea2fcbc3d8975"} Jul 13 17:32:36.633845 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:36.633808 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:32:36.719572 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:36.719532 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:32:36.821795 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:36.821752 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:32:36.909927 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:36.909883 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" event={"ID":"6bf45b3e-4198-45a9-8040-7a62ce3d07d1","Type":"ContainerStarted","Data":"785107a05144fef9fc91c0176ec5b92caa3779d72bbdce39ff5d8b4fe0ad0d42"} Jul 13 17:32:36.923346 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:36.923313 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:32:36.941716 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:36.941662 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" podStartSLOduration=1.941645759 podStartE2EDuration="1.941645759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:32:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:32:36.939617868 +0000 UTC m=+1634.602393057" watchObservedRunningTime="2026-07-13 17:32:36.941645759 +0000 UTC m=+1634.604420946" Jul 13 17:32:37.398327 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:37.398294 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:32:37.461377 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:37.461335 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:32:37.537183 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:37.537138 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:32:37.709159 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:37.709059 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:37.713920 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:37.713890 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:37.914873 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:37.914840 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:37.915910 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:37.915889 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:39.505862 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:39.505826 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:32:41.583537 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:41.583500 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq"] Jul 13 17:32:41.586157 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:41.586138 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq" Jul 13 17:32:41.589107 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:41.589087 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k25z5\"" Jul 13 17:32:41.589206 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:41.589128 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 17:32:41.593291 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:41.593266 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq"] Jul 13 17:32:41.746561 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:41.746523 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1759b965-7cb4-4cab-886e-8495f5cbfe7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-85580aa7-n96sq\" (UID: \"1759b965-7cb4-4cab-886e-8495f5cbfe7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq" Jul 13 17:32:41.847512 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:41.847405 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1759b965-7cb4-4cab-886e-8495f5cbfe7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-85580aa7-n96sq\" (UID: \"1759b965-7cb4-4cab-886e-8495f5cbfe7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq" Jul 13 17:32:41.848088 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:41.848063 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1759b965-7cb4-4cab-886e-8495f5cbfe7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-85580aa7-n96sq\" (UID: \"1759b965-7cb4-4cab-886e-8495f5cbfe7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq" Jul 13 17:32:41.898433 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:41.898396 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq" Jul 13 17:32:42.035953 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:42.035930 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq"] Jul 13 17:32:42.037861 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:32:42.037832 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1759b965_7cb4_4cab_886e_8495f5cbfe7a.slice/crio-30cf4cc5afb0cb9e639f7ec3f363f496dbe21d4779fecd45b97f6d5423a3cdac WatchSource:0}: Error finding container 30cf4cc5afb0cb9e639f7ec3f363f496dbe21d4779fecd45b97f6d5423a3cdac: Status 404 returned error can't find the container with id 30cf4cc5afb0cb9e639f7ec3f363f496dbe21d4779fecd45b97f6d5423a3cdac Jul 13 17:32:42.944615 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:42.944579 2564 generic.go:356] "Generic (PLEG): container finished" podID="1759b965-7cb4-4cab-886e-8495f5cbfe7a" containerID="a78fbcef44ae045083959599240ed90c528f83636be4838fcf51bf7fa3ad6c28" exitCode=7 Jul 13 17:32:42.945086 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:42.944666 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq" event={"ID":"1759b965-7cb4-4cab-886e-8495f5cbfe7a","Type":"ContainerDied","Data":"a78fbcef44ae045083959599240ed90c528f83636be4838fcf51bf7fa3ad6c28"} Jul 13 17:32:42.945086 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:42.944785 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq" event={"ID":"1759b965-7cb4-4cab-886e-8495f5cbfe7a","Type":"ContainerStarted","Data":"30cf4cc5afb0cb9e639f7ec3f363f496dbe21d4779fecd45b97f6d5423a3cdac"} Jul 13 17:32:42.945086 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:42.944835 2564 scope.go:117] "RemoveContainer" containerID="a78fbcef44ae045083959599240ed90c528f83636be4838fcf51bf7fa3ad6c28" Jul 13 17:32:43.951438 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:43.951346 2564 generic.go:356] "Generic (PLEG): container finished" podID="1759b965-7cb4-4cab-886e-8495f5cbfe7a" containerID="db699641e5de80f04c873707a630a02cd9efd383feb86497a5423bc6cd33f671" exitCode=7 Jul 13 17:32:43.951438 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:43.951428 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq" event={"ID":"1759b965-7cb4-4cab-886e-8495f5cbfe7a","Type":"ContainerDied","Data":"db699641e5de80f04c873707a630a02cd9efd383feb86497a5423bc6cd33f671"} Jul 13 17:32:43.951928 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:43.951482 2564 scope.go:117] "RemoveContainer" containerID="a78fbcef44ae045083959599240ed90c528f83636be4838fcf51bf7fa3ad6c28" Jul 13 17:32:43.951928 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:43.951647 2564 scope.go:117] "RemoveContainer" containerID="db699641e5de80f04c873707a630a02cd9efd383feb86497a5423bc6cd33f671" Jul 13 17:32:43.952021 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:32:43.951921 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-85580aa7-n96sq_odh-ai-gateway-infra(1759b965-7cb4-4cab-886e-8495f5cbfe7a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq" podUID="1759b965-7cb4-4cab-886e-8495f5cbfe7a" Jul 13 17:32:44.959642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:44.959612 2564 scope.go:117] "RemoveContainer" containerID="db699641e5de80f04c873707a630a02cd9efd383feb86497a5423bc6cd33f671" Jul 13 17:32:44.960151 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:32:44.959832 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-85580aa7-n96sq_odh-ai-gateway-infra(1759b965-7cb4-4cab-886e-8495f5cbfe7a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq" podUID="1759b965-7cb4-4cab-886e-8495f5cbfe7a" Jul 13 17:32:49.230333 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:49.230299 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:32:52.004616 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:52.004574 2564 generic.go:356] "Generic (PLEG): container finished" podID="6bf45b3e-4198-45a9-8040-7a62ce3d07d1" containerID="785107a05144fef9fc91c0176ec5b92caa3779d72bbdce39ff5d8b4fe0ad0d42" exitCode=0 Jul 13 17:32:52.005078 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:52.004638 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" event={"ID":"6bf45b3e-4198-45a9-8040-7a62ce3d07d1","Type":"ContainerDied","Data":"785107a05144fef9fc91c0176ec5b92caa3779d72bbdce39ff5d8b4fe0ad0d42"} Jul 13 17:32:52.005142 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:52.005095 2564 scope.go:117] "RemoveContainer" containerID="785107a05144fef9fc91c0176ec5b92caa3779d72bbdce39ff5d8b4fe0ad0d42" Jul 13 17:32:52.006160 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:52.006133 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:32:52.006254 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:52.006212 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:32:52.006316 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:52.006262 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:32:52.915273 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:52.915231 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:53.012186 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:53.012147 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" event={"ID":"6bf45b3e-4198-45a9-8040-7a62ce3d07d1","Type":"ContainerStarted","Data":"d7c07c0accafc1742b0fc552f134d7e749cf91bcafcd678df4bd672b84d621b6"} Jul 13 17:32:53.709561 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:53.709522 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:53.710694 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:53.710669 2564 patch_prober.go:28] interesting pod/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 13 17:32:53.710782 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:53.710739 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" podUID="6bf45b3e-4198-45a9-8040-7a62ce3d07d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 13 17:32:54.709076 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:54.709038 2564 patch_prober.go:28] interesting pod/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 13 17:32:54.709491 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:54.709108 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" podUID="6bf45b3e-4198-45a9-8040-7a62ce3d07d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 13 17:32:55.709437 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:55.709397 2564 patch_prober.go:28] interesting pod/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 13 17:32:55.709878 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:55.709471 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" podUID="6bf45b3e-4198-45a9-8040-7a62ce3d07d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 13 17:32:56.709515 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:56.709480 2564 patch_prober.go:28] interesting pod/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 13 17:32:56.709948 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:56.709552 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" podUID="6bf45b3e-4198-45a9-8040-7a62ce3d07d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 13 17:32:57.713914 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:57.713881 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:57.714398 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:57.714295 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:57.714912 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:57.714895 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:32:57.955586 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:57.955554 2564 scope.go:117] "RemoveContainer" containerID="db699641e5de80f04c873707a630a02cd9efd383feb86497a5423bc6cd33f671" Jul 13 17:32:59.045278 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:59.045188 2564 generic.go:356] "Generic (PLEG): container finished" podID="1759b965-7cb4-4cab-886e-8495f5cbfe7a" containerID="193374e44de0b75158eca61eb13a1d3ed7ee50999becebc46e4b54560a2c2cdb" exitCode=0 Jul 13 17:32:59.045278 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:59.045261 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq" event={"ID":"1759b965-7cb4-4cab-886e-8495f5cbfe7a","Type":"ContainerDied","Data":"193374e44de0b75158eca61eb13a1d3ed7ee50999becebc46e4b54560a2c2cdb"} Jul 13 17:32:59.045771 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:32:59.045333 2564 scope.go:117] "RemoveContainer" containerID="db699641e5de80f04c873707a630a02cd9efd383feb86497a5423bc6cd33f671" Jul 13 17:33:00.076177 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:00.076149 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq"] Jul 13 17:33:00.188589 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:00.188561 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq" Jul 13 17:33:00.230095 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:00.230065 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1759b965-7cb4-4cab-886e-8495f5cbfe7a-maas-api-service-ca\") pod \"1759b965-7cb4-4cab-886e-8495f5cbfe7a\" (UID: \"1759b965-7cb4-4cab-886e-8495f5cbfe7a\") " Jul 13 17:33:00.230446 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:00.230423 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1759b965-7cb4-4cab-886e-8495f5cbfe7a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1759b965-7cb4-4cab-886e-8495f5cbfe7a" (UID: "1759b965-7cb4-4cab-886e-8495f5cbfe7a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:33:00.331244 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:00.331156 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1759b965-7cb4-4cab-886e-8495f5cbfe7a-maas-api-service-ca\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:33:01.058544 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:01.058514 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq" Jul 13 17:33:01.058544 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:01.058536 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq" event={"ID":"1759b965-7cb4-4cab-886e-8495f5cbfe7a","Type":"ContainerDied","Data":"30cf4cc5afb0cb9e639f7ec3f363f496dbe21d4779fecd45b97f6d5423a3cdac"} Jul 13 17:33:01.058785 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:01.058578 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="30cf4cc5afb0cb9e639f7ec3f363f496dbe21d4779fecd45b97f6d5423a3cdac" Jul 13 17:33:01.091046 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:01.091016 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq"] Jul 13 17:33:01.099914 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:01.099888 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-n96sq"] Jul 13 17:33:02.960666 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:02.960629 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1759b965-7cb4-4cab-886e-8495f5cbfe7a" path="/var/lib/kubelet/pods/1759b965-7cb4-4cab-886e-8495f5cbfe7a/volumes" Jul 13 17:33:31.629035 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:31.629000 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-glfsk"] Jul 13 17:33:31.629528 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:31.629382 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1759b965-7cb4-4cab-886e-8495f5cbfe7a" containerName="revoke-keys" Jul 13 17:33:31.629528 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:31.629395 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1759b965-7cb4-4cab-886e-8495f5cbfe7a" containerName="revoke-keys" Jul 13 17:33:31.629528 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:31.629406 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1759b965-7cb4-4cab-886e-8495f5cbfe7a" containerName="revoke-keys" Jul 13 17:33:31.629528 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:31.629412 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1759b965-7cb4-4cab-886e-8495f5cbfe7a" containerName="revoke-keys" Jul 13 17:33:31.629528 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:31.629419 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1759b965-7cb4-4cab-886e-8495f5cbfe7a" containerName="revoke-keys" Jul 13 17:33:31.629528 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:31.629425 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1759b965-7cb4-4cab-886e-8495f5cbfe7a" containerName="revoke-keys" Jul 13 17:33:31.629528 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:31.629490 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1759b965-7cb4-4cab-886e-8495f5cbfe7a" containerName="revoke-keys" Jul 13 17:33:31.629528 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:31.629497 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1759b965-7cb4-4cab-886e-8495f5cbfe7a" containerName="revoke-keys" Jul 13 17:33:31.629528 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:31.629504 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1759b965-7cb4-4cab-886e-8495f5cbfe7a" containerName="revoke-keys" Jul 13 17:33:31.631625 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:31.631607 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-glfsk" Jul 13 17:33:31.634600 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:31.634581 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k25z5\"" Jul 13 17:33:31.634686 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:31.634581 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 17:33:31.639390 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:31.639323 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-glfsk"] Jul 13 17:33:31.713920 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:31.713885 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0f2f067-d17c-4ac1-bd10-a83de91c3d82-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-85580aa7-glfsk\" (UID: \"e0f2f067-d17c-4ac1-bd10-a83de91c3d82\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-glfsk" Jul 13 17:33:31.815201 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:31.815163 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0f2f067-d17c-4ac1-bd10-a83de91c3d82-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-85580aa7-glfsk\" (UID: \"e0f2f067-d17c-4ac1-bd10-a83de91c3d82\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-glfsk" Jul 13 17:33:31.815848 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:31.815826 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0f2f067-d17c-4ac1-bd10-a83de91c3d82-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-85580aa7-glfsk\" (UID: \"e0f2f067-d17c-4ac1-bd10-a83de91c3d82\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-glfsk" Jul 13 17:33:31.942577 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:31.942533 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-glfsk" Jul 13 17:33:32.074295 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:32.074265 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-glfsk"] Jul 13 17:33:32.075882 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:33:32.075853 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0f2f067_d17c_4ac1_bd10_a83de91c3d82.slice/crio-273ae4c4bd9f3cc28e891af06e9b6badbf29c2216dfb70710e5a46cf0820591f WatchSource:0}: Error finding container 273ae4c4bd9f3cc28e891af06e9b6badbf29c2216dfb70710e5a46cf0820591f: Status 404 returned error can't find the container with id 273ae4c4bd9f3cc28e891af06e9b6badbf29c2216dfb70710e5a46cf0820591f Jul 13 17:33:32.216879 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:32.216788 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-glfsk" event={"ID":"e0f2f067-d17c-4ac1-bd10-a83de91c3d82","Type":"ContainerStarted","Data":"273ae4c4bd9f3cc28e891af06e9b6badbf29c2216dfb70710e5a46cf0820591f"} Jul 13 17:33:33.223013 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:33.222979 2564 generic.go:356] "Generic (PLEG): container finished" podID="e0f2f067-d17c-4ac1-bd10-a83de91c3d82" containerID="cbe6ca37551016dae5fab73550fb54c68e9d75b90d04bf17286e61da34babe91" exitCode=0 Jul 13 17:33:33.223457 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:33.223069 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-glfsk" event={"ID":"e0f2f067-d17c-4ac1-bd10-a83de91c3d82","Type":"ContainerDied","Data":"cbe6ca37551016dae5fab73550fb54c68e9d75b90d04bf17286e61da34babe91"} Jul 13 17:33:34.361236 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:34.361214 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-glfsk" Jul 13 17:33:34.540236 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:34.540134 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0f2f067-d17c-4ac1-bd10-a83de91c3d82-maas-api-service-ca\") pod \"e0f2f067-d17c-4ac1-bd10-a83de91c3d82\" (UID: \"e0f2f067-d17c-4ac1-bd10-a83de91c3d82\") " Jul 13 17:33:34.540496 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:34.540473 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0f2f067-d17c-4ac1-bd10-a83de91c3d82-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e0f2f067-d17c-4ac1-bd10-a83de91c3d82" (UID: "e0f2f067-d17c-4ac1-bd10-a83de91c3d82"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:33:34.641595 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:34.641560 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0f2f067-d17c-4ac1-bd10-a83de91c3d82-maas-api-service-ca\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:33:35.234309 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:35.234225 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-glfsk" Jul 13 17:33:35.234309 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:35.234259 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-85580aa7-glfsk" event={"ID":"e0f2f067-d17c-4ac1-bd10-a83de91c3d82","Type":"ContainerDied","Data":"273ae4c4bd9f3cc28e891af06e9b6badbf29c2216dfb70710e5a46cf0820591f"} Jul 13 17:33:35.234309 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:35.234307 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="273ae4c4bd9f3cc28e891af06e9b6badbf29c2216dfb70710e5a46cf0820591f" Jul 13 17:33:42.099658 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:42.099615 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:33:45.201803 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:45.201767 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:33:46.713411 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:46.713376 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:33:52.774410 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:52.774374 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:33:52.873103 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:52.873057 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:33:56.910236 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:56.910189 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx"] Jul 13 17:33:56.910626 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:56.910467 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" podUID="6bf45b3e-4198-45a9-8040-7a62ce3d07d1" containerName="istio-proxy" containerID="cri-o://d7c07c0accafc1742b0fc552f134d7e749cf91bcafcd678df4bd672b84d621b6" gracePeriod=30 Jul 13 17:33:56.928142 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:56.928104 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:33:57.228793 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.228696 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9"] Jul 13 17:33:57.229271 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.229252 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0f2f067-d17c-4ac1-bd10-a83de91c3d82" containerName="revoke-keys" Jul 13 17:33:57.229322 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.229275 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0f2f067-d17c-4ac1-bd10-a83de91c3d82" containerName="revoke-keys" Jul 13 17:33:57.229388 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.229376 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0f2f067-d17c-4ac1-bd10-a83de91c3d82" containerName="revoke-keys" Jul 13 17:33:57.241623 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.241593 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.241830 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.241807 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9"] Jul 13 17:33:57.244650 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.244628 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0e7283f7-openshift-default-dockercfg-7h7fs\"" Jul 13 17:33:57.262220 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.262184 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:33:57.349763 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.349706 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-istio-envoy\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.349963 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.349837 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed272386-edb3-4e21-a30d-122218e304f9-istio-token\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.349963 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.349886 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-workload-socket\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.349963 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.349911 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj9mg\" (UniqueName: \"kubernetes.io/projected/ed272386-edb3-4e21-a30d-122218e304f9-kube-api-access-pj9mg\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.350134 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.349964 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-credential-socket\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.350134 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.349990 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-workload-certs\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.350134 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.350017 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-istio-data\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.350134 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.350056 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed272386-edb3-4e21-a30d-122218e304f9-istio-podinfo\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.350134 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.350127 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed272386-edb3-4e21-a30d-122218e304f9-istiod-ca-cert\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.450980 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.450941 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-credential-socket\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.450980 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.450980 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-workload-certs\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.451245 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.450999 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-istio-data\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.451245 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.451026 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed272386-edb3-4e21-a30d-122218e304f9-istio-podinfo\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.451245 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.451065 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed272386-edb3-4e21-a30d-122218e304f9-istiod-ca-cert\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.451245 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.451137 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-istio-envoy\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.451245 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.451167 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed272386-edb3-4e21-a30d-122218e304f9-istio-token\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.451245 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.451202 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-workload-socket\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.451245 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.451222 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj9mg\" (UniqueName: \"kubernetes.io/projected/ed272386-edb3-4e21-a30d-122218e304f9-kube-api-access-pj9mg\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.451577 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.451447 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-workload-certs\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.451701 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.451668 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-workload-socket\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.451701 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.451697 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-istio-data\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.452089 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.451765 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-credential-socket\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.452089 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.451980 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed272386-edb3-4e21-a30d-122218e304f9-istiod-ca-cert\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.453672 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.453650 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-istio-envoy\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.453860 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.453801 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed272386-edb3-4e21-a30d-122218e304f9-istio-podinfo\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.460205 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.460180 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed272386-edb3-4e21-a30d-122218e304f9-istio-token\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.460462 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.460436 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj9mg\" (UniqueName: \"kubernetes.io/projected/ed272386-edb3-4e21-a30d-122218e304f9-kube-api-access-pj9mg\") pod \"e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.552971 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.552871 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:57.699052 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.699024 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9"] Jul 13 17:33:57.700899 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:33:57.700858 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded272386_edb3_4e21_a30d_122218e304f9.slice/crio-5b60600f78acac988161e8a7fe52358b3a30aaf8bf0249ab9011aef71d05355d WatchSource:0}: Error finding container 5b60600f78acac988161e8a7fe52358b3a30aaf8bf0249ab9011aef71d05355d: Status 404 returned error can't find the container with id 5b60600f78acac988161e8a7fe52358b3a30aaf8bf0249ab9011aef71d05355d Jul 13 17:33:57.703173 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.703139 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:33:57.703302 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.703208 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:33:57.703302 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.703259 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:33:57.716411 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.716379 2564 patch_prober.go:28] interesting pod/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 17:33:57.716535 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:57.716455 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" podUID="6bf45b3e-4198-45a9-8040-7a62ce3d07d1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 17:33:58.367207 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:58.367173 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" event={"ID":"ed272386-edb3-4e21-a30d-122218e304f9","Type":"ContainerStarted","Data":"c92e9e92a0a470c91960e2558f4c1e882f82971f21bd7debe82a0c0f5dd7840b"} Jul 13 17:33:58.367644 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:58.367222 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" event={"ID":"ed272386-edb3-4e21-a30d-122218e304f9","Type":"ContainerStarted","Data":"5b60600f78acac988161e8a7fe52358b3a30aaf8bf0249ab9011aef71d05355d"} Jul 13 17:33:58.390009 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:58.389961 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" podStartSLOduration=1.3899460989999999 podStartE2EDuration="1.389946099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:33:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:33:58.387978451 +0000 UTC m=+1716.050753641" watchObservedRunningTime="2026-07-13 17:33:58.389946099 +0000 UTC m=+1716.052721287" Jul 13 17:33:58.479924 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:58.479887 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:33:58.553045 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:58.553010 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:58.558706 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:58.558680 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:58.572225 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:58.572191 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:33:58.642861 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:58.642780 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:33:59.066980 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:59.066941 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:33:59.166997 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:59.166954 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:33:59.267462 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:59.267376 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:33:59.372832 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:59.372709 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:33:59.373877 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:33:59.373855 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:34:01.128218 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:01.128182 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:34:02.169361 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.169039 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:34:02.306897 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.306863 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-envoy\") pod \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " Jul 13 17:34:02.307103 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.306958 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-workload-certs\") pod \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " Jul 13 17:34:02.307103 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.307012 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-data\") pod \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " Jul 13 17:34:02.307103 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.307037 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istiod-ca-cert\") pod \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " Jul 13 17:34:02.307103 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.307071 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-token\") pod \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " Jul 13 17:34:02.307312 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.307104 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wr9v6\" (UniqueName: \"kubernetes.io/projected/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-kube-api-access-wr9v6\") pod \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " Jul 13 17:34:02.307312 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.307134 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-credential-socket\") pod \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " Jul 13 17:34:02.307312 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.307161 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-workload-socket\") pod \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " Jul 13 17:34:02.307312 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.307199 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-podinfo\") pod \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\" (UID: \"6bf45b3e-4198-45a9-8040-7a62ce3d07d1\") " Jul 13 17:34:02.307312 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.307229 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6bf45b3e-4198-45a9-8040-7a62ce3d07d1" (UID: "6bf45b3e-4198-45a9-8040-7a62ce3d07d1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:34:02.307550 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.307414 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6bf45b3e-4198-45a9-8040-7a62ce3d07d1" (UID: "6bf45b3e-4198-45a9-8040-7a62ce3d07d1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:34:02.307550 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.307532 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-workload-certs\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:34:02.307654 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.307554 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istiod-ca-cert\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:34:02.308171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.307719 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6bf45b3e-4198-45a9-8040-7a62ce3d07d1" (UID: "6bf45b3e-4198-45a9-8040-7a62ce3d07d1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:34:02.308171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.307843 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-data" (OuterVolumeSpecName: "istio-data") pod "6bf45b3e-4198-45a9-8040-7a62ce3d07d1" (UID: "6bf45b3e-4198-45a9-8040-7a62ce3d07d1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:34:02.308171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.308120 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6bf45b3e-4198-45a9-8040-7a62ce3d07d1" (UID: "6bf45b3e-4198-45a9-8040-7a62ce3d07d1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:34:02.309544 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.309508 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-kube-api-access-wr9v6" (OuterVolumeSpecName: "kube-api-access-wr9v6") pod "6bf45b3e-4198-45a9-8040-7a62ce3d07d1" (UID: "6bf45b3e-4198-45a9-8040-7a62ce3d07d1"). InnerVolumeSpecName "kube-api-access-wr9v6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:34:02.309643 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.309548 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-token" (OuterVolumeSpecName: "istio-token") pod "6bf45b3e-4198-45a9-8040-7a62ce3d07d1" (UID: "6bf45b3e-4198-45a9-8040-7a62ce3d07d1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:34:02.309643 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.309588 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6bf45b3e-4198-45a9-8040-7a62ce3d07d1" (UID: "6bf45b3e-4198-45a9-8040-7a62ce3d07d1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:34:02.309643 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.309605 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6bf45b3e-4198-45a9-8040-7a62ce3d07d1" (UID: "6bf45b3e-4198-45a9-8040-7a62ce3d07d1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:34:02.395756 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.395596 2564 generic.go:356] "Generic (PLEG): container finished" podID="6bf45b3e-4198-45a9-8040-7a62ce3d07d1" containerID="d7c07c0accafc1742b0fc552f134d7e749cf91bcafcd678df4bd672b84d621b6" exitCode=0 Jul 13 17:34:02.395988 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.395856 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" event={"ID":"6bf45b3e-4198-45a9-8040-7a62ce3d07d1","Type":"ContainerDied","Data":"d7c07c0accafc1742b0fc552f134d7e749cf91bcafcd678df4bd672b84d621b6"} Jul 13 17:34:02.395988 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.395914 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" event={"ID":"6bf45b3e-4198-45a9-8040-7a62ce3d07d1","Type":"ContainerDied","Data":"611c0eea6a56502d5814ea0ad265827b01b7019893a16efa99aea2fcbc3d8975"} Jul 13 17:34:02.395988 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.395945 2564 scope.go:117] "RemoveContainer" containerID="d7c07c0accafc1742b0fc552f134d7e749cf91bcafcd678df4bd672b84d621b6" Jul 13 17:34:02.396170 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.396038 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx" Jul 13 17:34:02.407264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.407243 2564 scope.go:117] "RemoveContainer" containerID="785107a05144fef9fc91c0176ec5b92caa3779d72bbdce39ff5d8b4fe0ad0d42" Jul 13 17:34:02.408010 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.407990 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-workload-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:34:02.408080 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.408013 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-podinfo\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:34:02.408080 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.408026 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-envoy\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:34:02.408080 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.408035 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-data\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:34:02.408080 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.408043 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-istio-token\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:34:02.408080 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.408052 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wr9v6\" (UniqueName: \"kubernetes.io/projected/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-kube-api-access-wr9v6\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:34:02.408080 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.408065 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bf45b3e-4198-45a9-8040-7a62ce3d07d1-credential-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:34:02.415695 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.415671 2564 scope.go:117] "RemoveContainer" containerID="d7c07c0accafc1742b0fc552f134d7e749cf91bcafcd678df4bd672b84d621b6" Jul 13 17:34:02.416009 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:34:02.415990 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7c07c0accafc1742b0fc552f134d7e749cf91bcafcd678df4bd672b84d621b6\": container with ID starting with d7c07c0accafc1742b0fc552f134d7e749cf91bcafcd678df4bd672b84d621b6 not found: ID does not exist" containerID="d7c07c0accafc1742b0fc552f134d7e749cf91bcafcd678df4bd672b84d621b6" Jul 13 17:34:02.416068 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.416020 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7c07c0accafc1742b0fc552f134d7e749cf91bcafcd678df4bd672b84d621b6"} err="failed to get container status \"d7c07c0accafc1742b0fc552f134d7e749cf91bcafcd678df4bd672b84d621b6\": rpc error: code = NotFound desc = could not find container \"d7c07c0accafc1742b0fc552f134d7e749cf91bcafcd678df4bd672b84d621b6\": container with ID starting with d7c07c0accafc1742b0fc552f134d7e749cf91bcafcd678df4bd672b84d621b6 not found: ID does not exist" Jul 13 17:34:02.416068 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.416038 2564 scope.go:117] "RemoveContainer" containerID="785107a05144fef9fc91c0176ec5b92caa3779d72bbdce39ff5d8b4fe0ad0d42" Jul 13 17:34:02.416292 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:34:02.416271 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"785107a05144fef9fc91c0176ec5b92caa3779d72bbdce39ff5d8b4fe0ad0d42\": container with ID starting with 785107a05144fef9fc91c0176ec5b92caa3779d72bbdce39ff5d8b4fe0ad0d42 not found: ID does not exist" containerID="785107a05144fef9fc91c0176ec5b92caa3779d72bbdce39ff5d8b4fe0ad0d42" Jul 13 17:34:02.416351 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.416303 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"785107a05144fef9fc91c0176ec5b92caa3779d72bbdce39ff5d8b4fe0ad0d42"} err="failed to get container status \"785107a05144fef9fc91c0176ec5b92caa3779d72bbdce39ff5d8b4fe0ad0d42\": rpc error: code = NotFound desc = could not find container \"785107a05144fef9fc91c0176ec5b92caa3779d72bbdce39ff5d8b4fe0ad0d42\": container with ID starting with 785107a05144fef9fc91c0176ec5b92caa3779d72bbdce39ff5d8b4fe0ad0d42 not found: ID does not exist" Jul 13 17:34:02.420780 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.420754 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx"] Jul 13 17:34:02.425134 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.425107 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-85580aa7-openshift-default-8f6f44f65-s2vkx"] Jul 13 17:34:02.962252 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:02.962214 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bf45b3e-4198-45a9-8040-7a62ce3d07d1" path="/var/lib/kubelet/pods/6bf45b3e-4198-45a9-8040-7a62ce3d07d1/volumes" Jul 13 17:34:03.418456 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:03.418426 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:34:03.496334 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:03.496286 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:34:03.580909 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:03.580868 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr"] Jul 13 17:34:03.581447 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:03.581423 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf45b3e-4198-45a9-8040-7a62ce3d07d1" containerName="istio-proxy" Jul 13 17:34:03.581635 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:03.581450 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf45b3e-4198-45a9-8040-7a62ce3d07d1" containerName="istio-proxy" Jul 13 17:34:03.581635 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:03.581471 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf45b3e-4198-45a9-8040-7a62ce3d07d1" containerName="istio-proxy" Jul 13 17:34:03.581635 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:03.581481 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf45b3e-4198-45a9-8040-7a62ce3d07d1" containerName="istio-proxy" Jul 13 17:34:03.581635 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:03.581566 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bf45b3e-4198-45a9-8040-7a62ce3d07d1" containerName="istio-proxy" Jul 13 17:34:03.585530 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:03.585498 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr" Jul 13 17:34:03.588465 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:03.588430 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k25z5\"" Jul 13 17:34:03.588768 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:03.588747 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 17:34:03.601097 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:03.601059 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr"] Jul 13 17:34:03.721038 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:03.720947 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/823ba6c7-e2a8-487b-9989-36cd7d2f60ce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr\" (UID: \"823ba6c7-e2a8-487b-9989-36cd7d2f60ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr" Jul 13 17:34:03.822619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:03.822578 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/823ba6c7-e2a8-487b-9989-36cd7d2f60ce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr\" (UID: \"823ba6c7-e2a8-487b-9989-36cd7d2f60ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr" Jul 13 17:34:03.823264 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:03.823240 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/823ba6c7-e2a8-487b-9989-36cd7d2f60ce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr\" (UID: \"823ba6c7-e2a8-487b-9989-36cd7d2f60ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr" Jul 13 17:34:03.902589 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:03.902546 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr" Jul 13 17:34:04.062745 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:04.062702 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr"] Jul 13 17:34:04.064609 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:34:04.064576 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod823ba6c7_e2a8_487b_9989_36cd7d2f60ce.slice/crio-19cb04ba618fac9b4acbdd6f39e12386fa01ed78a990007fd2429b46f140126d WatchSource:0}: Error finding container 19cb04ba618fac9b4acbdd6f39e12386fa01ed78a990007fd2429b46f140126d: Status 404 returned error can't find the container with id 19cb04ba618fac9b4acbdd6f39e12386fa01ed78a990007fd2429b46f140126d Jul 13 17:34:04.411512 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:04.411473 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr" event={"ID":"823ba6c7-e2a8-487b-9989-36cd7d2f60ce","Type":"ContainerStarted","Data":"87975b689171711b978e95b11df355d261772175648bfee2b2dfa3a303ce818d"} Jul 13 17:34:04.411648 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:04.411556 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr" event={"ID":"823ba6c7-e2a8-487b-9989-36cd7d2f60ce","Type":"ContainerStarted","Data":"19cb04ba618fac9b4acbdd6f39e12386fa01ed78a990007fd2429b46f140126d"} Jul 13 17:34:04.430248 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:04.430195 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr" podStartSLOduration=1.430175993 podStartE2EDuration="1.430175993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:34:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:34:04.428931014 +0000 UTC m=+1722.091706202" watchObservedRunningTime="2026-07-13 17:34:04.430175993 +0000 UTC m=+1722.092951184" Jul 13 17:34:05.418382 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:05.418343 2564 generic.go:356] "Generic (PLEG): container finished" podID="823ba6c7-e2a8-487b-9989-36cd7d2f60ce" containerID="87975b689171711b978e95b11df355d261772175648bfee2b2dfa3a303ce818d" exitCode=7 Jul 13 17:34:05.418580 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:05.418391 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr" event={"ID":"823ba6c7-e2a8-487b-9989-36cd7d2f60ce","Type":"ContainerDied","Data":"87975b689171711b978e95b11df355d261772175648bfee2b2dfa3a303ce818d"} Jul 13 17:34:05.418648 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:05.418633 2564 scope.go:117] "RemoveContainer" containerID="87975b689171711b978e95b11df355d261772175648bfee2b2dfa3a303ce818d" Jul 13 17:34:06.427220 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:06.427179 2564 generic.go:356] "Generic (PLEG): container finished" podID="823ba6c7-e2a8-487b-9989-36cd7d2f60ce" containerID="d800ccee84283758a4e08c7b1fab358a6532160f84903f3ce37bccf2c1594061" exitCode=7 Jul 13 17:34:06.427624 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:06.427261 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr" event={"ID":"823ba6c7-e2a8-487b-9989-36cd7d2f60ce","Type":"ContainerDied","Data":"d800ccee84283758a4e08c7b1fab358a6532160f84903f3ce37bccf2c1594061"} Jul 13 17:34:06.427624 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:06.427329 2564 scope.go:117] "RemoveContainer" containerID="87975b689171711b978e95b11df355d261772175648bfee2b2dfa3a303ce818d" Jul 13 17:34:06.427624 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:06.427523 2564 scope.go:117] "RemoveContainer" containerID="d800ccee84283758a4e08c7b1fab358a6532160f84903f3ce37bccf2c1594061" Jul 13 17:34:06.427848 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:34:06.427819 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr_odh-ai-gateway-infra(823ba6c7-e2a8-487b-9989-36cd7d2f60ce)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr" podUID="823ba6c7-e2a8-487b-9989-36cd7d2f60ce" Jul 13 17:34:07.369936 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:07.369855 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:34:07.437436 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:07.437399 2564 scope.go:117] "RemoveContainer" containerID="d800ccee84283758a4e08c7b1fab358a6532160f84903f3ce37bccf2c1594061" Jul 13 17:34:07.437962 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:34:07.437630 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr_odh-ai-gateway-infra(823ba6c7-e2a8-487b-9989-36cd7d2f60ce)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr" podUID="823ba6c7-e2a8-487b-9989-36cd7d2f60ce" Jul 13 17:34:09.450028 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:09.449939 2564 generic.go:356] "Generic (PLEG): container finished" podID="ed272386-edb3-4e21-a30d-122218e304f9" containerID="c92e9e92a0a470c91960e2558f4c1e882f82971f21bd7debe82a0c0f5dd7840b" exitCode=0 Jul 13 17:34:09.450028 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:09.450008 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" event={"ID":"ed272386-edb3-4e21-a30d-122218e304f9","Type":"ContainerDied","Data":"c92e9e92a0a470c91960e2558f4c1e882f82971f21bd7debe82a0c0f5dd7840b"} Jul 13 17:34:09.450478 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:09.450410 2564 scope.go:117] "RemoveContainer" containerID="c92e9e92a0a470c91960e2558f4c1e882f82971f21bd7debe82a0c0f5dd7840b" Jul 13 17:34:09.451421 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:09.451398 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:34:09.451488 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:09.451476 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:34:09.451526 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:09.451508 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:34:10.457553 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:10.457516 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" event={"ID":"ed272386-edb3-4e21-a30d-122218e304f9","Type":"ContainerStarted","Data":"8300bcb377d188d3efea246efd1d01a0d7dc9cba05a84b898a69977b0744c816"} Jul 13 17:34:10.553334 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:10.553293 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:34:10.554272 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:10.554250 2564 patch_prober.go:28] interesting pod/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 13 17:34:10.554458 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:10.554296 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" podUID="ed272386-edb3-4e21-a30d-122218e304f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 13 17:34:11.553487 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:11.553441 2564 patch_prober.go:28] interesting pod/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 13 17:34:11.553906 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:11.553508 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" podUID="ed272386-edb3-4e21-a30d-122218e304f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 13 17:34:12.554036 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:12.553997 2564 patch_prober.go:28] interesting pod/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 13 17:34:12.554433 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:12.554058 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" podUID="ed272386-edb3-4e21-a30d-122218e304f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 13 17:34:13.554314 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:13.554275 2564 patch_prober.go:28] interesting pod/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 13 17:34:13.554708 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:13.554336 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" podUID="ed272386-edb3-4e21-a30d-122218e304f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 13 17:34:14.373624 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:14.373580 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:34:14.556858 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:14.556828 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:34:14.557896 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:14.557877 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:34:18.955369 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:18.955334 2564 scope.go:117] "RemoveContainer" containerID="d800ccee84283758a4e08c7b1fab358a6532160f84903f3ce37bccf2c1594061" Jul 13 17:34:19.518469 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:19.518433 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr" event={"ID":"823ba6c7-e2a8-487b-9989-36cd7d2f60ce","Type":"ContainerStarted","Data":"82166d52749cc3f0d559f072ddf90090d1cf1d6b2a212ecbd27fceffbd2d9d1a"} Jul 13 17:34:20.525621 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:20.525587 2564 generic.go:356] "Generic (PLEG): container finished" podID="823ba6c7-e2a8-487b-9989-36cd7d2f60ce" containerID="82166d52749cc3f0d559f072ddf90090d1cf1d6b2a212ecbd27fceffbd2d9d1a" exitCode=0 Jul 13 17:34:20.526078 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:20.525663 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr" event={"ID":"823ba6c7-e2a8-487b-9989-36cd7d2f60ce","Type":"ContainerDied","Data":"82166d52749cc3f0d559f072ddf90090d1cf1d6b2a212ecbd27fceffbd2d9d1a"} Jul 13 17:34:20.526078 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:20.525739 2564 scope.go:117] "RemoveContainer" containerID="d800ccee84283758a4e08c7b1fab358a6532160f84903f3ce37bccf2c1594061" Jul 13 17:34:20.550510 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:20.550469 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr"] Jul 13 17:34:21.670183 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:21.670152 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr" Jul 13 17:34:21.799126 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:21.799045 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/823ba6c7-e2a8-487b-9989-36cd7d2f60ce-maas-api-service-ca\") pod \"823ba6c7-e2a8-487b-9989-36cd7d2f60ce\" (UID: \"823ba6c7-e2a8-487b-9989-36cd7d2f60ce\") " Jul 13 17:34:21.799398 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:21.799375 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/823ba6c7-e2a8-487b-9989-36cd7d2f60ce-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "823ba6c7-e2a8-487b-9989-36cd7d2f60ce" (UID: "823ba6c7-e2a8-487b-9989-36cd7d2f60ce"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:34:21.900265 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:21.900233 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/823ba6c7-e2a8-487b-9989-36cd7d2f60ce-maas-api-service-ca\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:34:22.539156 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:22.539123 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr" Jul 13 17:34:22.539367 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:22.539126 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr" event={"ID":"823ba6c7-e2a8-487b-9989-36cd7d2f60ce","Type":"ContainerDied","Data":"19cb04ba618fac9b4acbdd6f39e12386fa01ed78a990007fd2429b46f140126d"} Jul 13 17:34:22.539367 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:22.539244 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19cb04ba618fac9b4acbdd6f39e12386fa01ed78a990007fd2429b46f140126d" Jul 13 17:34:22.570788 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:22.570757 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr"] Jul 13 17:34:22.574666 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:22.574639 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-sd5xr"] Jul 13 17:34:22.962248 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:22.962213 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="823ba6c7-e2a8-487b-9989-36cd7d2f60ce" path="/var/lib/kubelet/pods/823ba6c7-e2a8-487b-9989-36cd7d2f60ce/volumes" Jul 13 17:34:53.607835 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:53.607803 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-kp787"] Jul 13 17:34:53.608298 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:53.608136 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="823ba6c7-e2a8-487b-9989-36cd7d2f60ce" containerName="revoke-keys" Jul 13 17:34:53.608298 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:53.608146 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="823ba6c7-e2a8-487b-9989-36cd7d2f60ce" containerName="revoke-keys" Jul 13 17:34:53.608298 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:53.608154 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="823ba6c7-e2a8-487b-9989-36cd7d2f60ce" containerName="revoke-keys" Jul 13 17:34:53.608298 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:53.608159 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="823ba6c7-e2a8-487b-9989-36cd7d2f60ce" containerName="revoke-keys" Jul 13 17:34:53.608298 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:53.608240 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="823ba6c7-e2a8-487b-9989-36cd7d2f60ce" containerName="revoke-keys" Jul 13 17:34:53.608298 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:53.608250 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bf45b3e-4198-45a9-8040-7a62ce3d07d1" containerName="istio-proxy" Jul 13 17:34:53.608298 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:53.608256 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="823ba6c7-e2a8-487b-9989-36cd7d2f60ce" containerName="revoke-keys" Jul 13 17:34:53.610417 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:53.610400 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-kp787" Jul 13 17:34:53.613550 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:53.613530 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 17:34:53.613681 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:53.613665 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k25z5\"" Jul 13 17:34:53.619743 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:53.619708 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-kp787"] Jul 13 17:34:53.674942 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:53.674909 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eea6d6e-c248-460f-a3d8-447bbd419ebb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0e7283f7-kp787\" (UID: \"0eea6d6e-c248-460f-a3d8-447bbd419ebb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-kp787" Jul 13 17:34:53.775973 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:53.775935 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eea6d6e-c248-460f-a3d8-447bbd419ebb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0e7283f7-kp787\" (UID: \"0eea6d6e-c248-460f-a3d8-447bbd419ebb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-kp787" Jul 13 17:34:53.776622 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:53.776602 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eea6d6e-c248-460f-a3d8-447bbd419ebb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0e7283f7-kp787\" (UID: \"0eea6d6e-c248-460f-a3d8-447bbd419ebb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-kp787" Jul 13 17:34:53.921495 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:53.921464 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-kp787" Jul 13 17:34:54.056316 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:54.056290 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-kp787"] Jul 13 17:34:54.058246 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:34:54.058222 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eea6d6e_c248_460f_a3d8_447bbd419ebb.slice/crio-e0cb18284e6f46ba8601a261538c95a72b99c69246b5fe0e4df20d8e4ee3bf7a WatchSource:0}: Error finding container e0cb18284e6f46ba8601a261538c95a72b99c69246b5fe0e4df20d8e4ee3bf7a: Status 404 returned error can't find the container with id e0cb18284e6f46ba8601a261538c95a72b99c69246b5fe0e4df20d8e4ee3bf7a Jul 13 17:34:54.706437 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:54.706407 2564 generic.go:356] "Generic (PLEG): container finished" podID="0eea6d6e-c248-460f-a3d8-447bbd419ebb" containerID="f84d413d0021ae4e603258943c23eebfde3454115401e4a912294be5ffa2cbe2" exitCode=0 Jul 13 17:34:54.706797 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:54.706468 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-kp787" event={"ID":"0eea6d6e-c248-460f-a3d8-447bbd419ebb","Type":"ContainerDied","Data":"f84d413d0021ae4e603258943c23eebfde3454115401e4a912294be5ffa2cbe2"} Jul 13 17:34:54.706797 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:54.706506 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-kp787" event={"ID":"0eea6d6e-c248-460f-a3d8-447bbd419ebb","Type":"ContainerStarted","Data":"e0cb18284e6f46ba8601a261538c95a72b99c69246b5fe0e4df20d8e4ee3bf7a"} Jul 13 17:34:55.851268 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:55.851243 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-kp787" Jul 13 17:34:55.997245 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:55.997150 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eea6d6e-c248-460f-a3d8-447bbd419ebb-maas-api-service-ca\") pod \"0eea6d6e-c248-460f-a3d8-447bbd419ebb\" (UID: \"0eea6d6e-c248-460f-a3d8-447bbd419ebb\") " Jul 13 17:34:55.997532 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:55.997508 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eea6d6e-c248-460f-a3d8-447bbd419ebb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0eea6d6e-c248-460f-a3d8-447bbd419ebb" (UID: "0eea6d6e-c248-460f-a3d8-447bbd419ebb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:34:56.098528 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:56.098491 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eea6d6e-c248-460f-a3d8-447bbd419ebb-maas-api-service-ca\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:34:56.718514 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:56.718477 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-kp787" event={"ID":"0eea6d6e-c248-460f-a3d8-447bbd419ebb","Type":"ContainerDied","Data":"e0cb18284e6f46ba8601a261538c95a72b99c69246b5fe0e4df20d8e4ee3bf7a"} Jul 13 17:34:56.718750 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:56.718534 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0cb18284e6f46ba8601a261538c95a72b99c69246b5fe0e4df20d8e4ee3bf7a" Jul 13 17:34:56.718750 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:34:56.718486 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e7283f7-kp787" Jul 13 17:35:04.187241 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:04.187196 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:07.239685 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:07.239647 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:35:08.800627 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:08.800582 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:14.429377 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:14.429294 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:14.494207 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:14.494170 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:19.728176 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:19.728144 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9"] Jul 13 17:35:19.728651 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:19.728354 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" podUID="ed272386-edb3-4e21-a30d-122218e304f9" containerName="istio-proxy" containerID="cri-o://8300bcb377d188d3efea246efd1d01a0d7dc9cba05a84b898a69977b0744c816" gracePeriod=30 Jul 13 17:35:19.751365 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:19.751308 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:20.013140 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.013052 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s"] Jul 13 17:35:20.013590 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.013572 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="823ba6c7-e2a8-487b-9989-36cd7d2f60ce" containerName="revoke-keys" Jul 13 17:35:20.013683 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.013592 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="823ba6c7-e2a8-487b-9989-36cd7d2f60ce" containerName="revoke-keys" Jul 13 17:35:20.013683 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.013628 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0eea6d6e-c248-460f-a3d8-447bbd419ebb" containerName="revoke-keys" Jul 13 17:35:20.013683 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.013637 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0eea6d6e-c248-460f-a3d8-447bbd419ebb" containerName="revoke-keys" Jul 13 17:35:20.013886 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.013710 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="823ba6c7-e2a8-487b-9989-36cd7d2f60ce" containerName="revoke-keys" Jul 13 17:35:20.013886 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.013745 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0eea6d6e-c248-460f-a3d8-447bbd419ebb" containerName="revoke-keys" Jul 13 17:35:20.016696 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.016670 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.019579 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.019558 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-8e6203f1-openshift-default-dockercfg-ffxwk\"" Jul 13 17:35:20.028009 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.027983 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s"] Jul 13 17:35:20.056083 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.056045 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:20.117019 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.116986 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-data\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.117200 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.117078 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-token\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.117200 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.117132 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-workload-socket\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.117200 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.117150 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-credential-socket\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.117200 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.117177 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-envoy\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.117357 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.117225 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-podinfo\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.117357 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.117241 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbh9w\" (UniqueName: \"kubernetes.io/projected/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-kube-api-access-xbh9w\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.117357 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.117261 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-workload-certs\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.117357 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.117297 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istiod-ca-cert\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.218193 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.218153 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-data\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.218386 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.218214 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-token\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.218386 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.218249 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-workload-socket\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.218386 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.218270 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-credential-socket\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.218386 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.218295 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-envoy\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.218680 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.218450 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-podinfo\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.218680 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.218498 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbh9w\" (UniqueName: \"kubernetes.io/projected/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-kube-api-access-xbh9w\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.218680 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.218534 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-workload-certs\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.218680 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.218563 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istiod-ca-cert\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.218680 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.218570 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-data\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.218989 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.218794 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-credential-socket\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.218989 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.218845 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-workload-certs\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.219097 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.219043 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-workload-socket\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.219193 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.219175 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istiod-ca-cert\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.220800 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.220774 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-envoy\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.221010 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.220991 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-podinfo\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.227984 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.227950 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-token\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.228106 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.228086 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbh9w\" (UniqueName: \"kubernetes.io/projected/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-kube-api-access-xbh9w\") pod \"e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.331787 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.331675 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:20.481300 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.481268 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s"] Jul 13 17:35:20.483495 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:35:20.483465 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf4ac151_c14a_42b2_bb39_c5d5f4c47ea8.slice/crio-e9af95d452d43caeda4d5e91edc64d7b5de731221a0ca5da30c5c1a34f1d7906 WatchSource:0}: Error finding container e9af95d452d43caeda4d5e91edc64d7b5de731221a0ca5da30c5c1a34f1d7906: Status 404 returned error can't find the container with id e9af95d452d43caeda4d5e91edc64d7b5de731221a0ca5da30c5c1a34f1d7906 Jul 13 17:35:20.486060 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.486026 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:35:20.486149 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.486092 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:35:20.486149 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.486127 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:35:20.857204 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.857165 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" event={"ID":"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8","Type":"ContainerStarted","Data":"58f5536c50686a86c9cacea3bde0bc76915552218e2e4cf9d431fcadfb2f8425"} Jul 13 17:35:20.857616 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.857221 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" event={"ID":"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8","Type":"ContainerStarted","Data":"e9af95d452d43caeda4d5e91edc64d7b5de731221a0ca5da30c5c1a34f1d7906"} Jul 13 17:35:20.880274 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:20.880221 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" podStartSLOduration=1.88020375 podStartE2EDuration="1.88020375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:35:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:35:20.879348198 +0000 UTC m=+1798.542123386" watchObservedRunningTime="2026-07-13 17:35:20.88020375 +0000 UTC m=+1798.542978937" Jul 13 17:35:21.168472 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:21.168433 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:21.327559 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:21.327524 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:21.332629 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:21.332605 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:21.337372 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:21.337344 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:21.486449 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:21.486348 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:21.584262 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:21.584231 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:21.862683 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:21.862596 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:21.863631 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:21.863614 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:21.877663 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:21.877633 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:21.974393 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:21.974361 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:24.126071 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:24.126025 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:35:24.884303 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:24.884267 2564 generic.go:356] "Generic (PLEG): container finished" podID="ed272386-edb3-4e21-a30d-122218e304f9" containerID="8300bcb377d188d3efea246efd1d01a0d7dc9cba05a84b898a69977b0744c816" exitCode=0 Jul 13 17:35:24.884514 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:24.884329 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" event={"ID":"ed272386-edb3-4e21-a30d-122218e304f9","Type":"ContainerDied","Data":"8300bcb377d188d3efea246efd1d01a0d7dc9cba05a84b898a69977b0744c816"} Jul 13 17:35:24.884514 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:24.884395 2564 scope.go:117] "RemoveContainer" containerID="c92e9e92a0a470c91960e2558f4c1e882f82971f21bd7debe82a0c0f5dd7840b" Jul 13 17:35:24.986165 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:24.986141 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:35:25.065662 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.065626 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-workload-socket\") pod \"ed272386-edb3-4e21-a30d-122218e304f9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " Jul 13 17:35:25.065877 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.065694 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-credential-socket\") pod \"ed272386-edb3-4e21-a30d-122218e304f9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " Jul 13 17:35:25.065877 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.065765 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-istio-envoy\") pod \"ed272386-edb3-4e21-a30d-122218e304f9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " Jul 13 17:35:25.065877 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.065792 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-workload-certs\") pod \"ed272386-edb3-4e21-a30d-122218e304f9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " Jul 13 17:35:25.065877 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.065850 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-istio-data\") pod \"ed272386-edb3-4e21-a30d-122218e304f9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " Jul 13 17:35:25.065877 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.065877 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed272386-edb3-4e21-a30d-122218e304f9-istio-podinfo\") pod \"ed272386-edb3-4e21-a30d-122218e304f9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " Jul 13 17:35:25.066153 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.065903 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed272386-edb3-4e21-a30d-122218e304f9-istiod-ca-cert\") pod \"ed272386-edb3-4e21-a30d-122218e304f9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " Jul 13 17:35:25.066153 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.065916 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ed272386-edb3-4e21-a30d-122218e304f9" (UID: "ed272386-edb3-4e21-a30d-122218e304f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:35:25.066153 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.065953 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pj9mg\" (UniqueName: \"kubernetes.io/projected/ed272386-edb3-4e21-a30d-122218e304f9-kube-api-access-pj9mg\") pod \"ed272386-edb3-4e21-a30d-122218e304f9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " Jul 13 17:35:25.066153 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.065988 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed272386-edb3-4e21-a30d-122218e304f9-istio-token\") pod \"ed272386-edb3-4e21-a30d-122218e304f9\" (UID: \"ed272386-edb3-4e21-a30d-122218e304f9\") " Jul 13 17:35:25.066153 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.066090 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ed272386-edb3-4e21-a30d-122218e304f9" (UID: "ed272386-edb3-4e21-a30d-122218e304f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:35:25.066713 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.066622 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed272386-edb3-4e21-a30d-122218e304f9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ed272386-edb3-4e21-a30d-122218e304f9" (UID: "ed272386-edb3-4e21-a30d-122218e304f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:35:25.066985 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.066915 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-workload-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:35:25.066985 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.066942 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-credential-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:35:25.066985 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.066957 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed272386-edb3-4e21-a30d-122218e304f9-istiod-ca-cert\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:35:25.067188 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.067042 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-istio-data" (OuterVolumeSpecName: "istio-data") pod "ed272386-edb3-4e21-a30d-122218e304f9" (UID: "ed272386-edb3-4e21-a30d-122218e304f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:35:25.067643 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.067607 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ed272386-edb3-4e21-a30d-122218e304f9" (UID: "ed272386-edb3-4e21-a30d-122218e304f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:35:25.068858 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.068819 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed272386-edb3-4e21-a30d-122218e304f9-istio-token" (OuterVolumeSpecName: "istio-token") pod "ed272386-edb3-4e21-a30d-122218e304f9" (UID: "ed272386-edb3-4e21-a30d-122218e304f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:35:25.069064 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.069043 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ed272386-edb3-4e21-a30d-122218e304f9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ed272386-edb3-4e21-a30d-122218e304f9" (UID: "ed272386-edb3-4e21-a30d-122218e304f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:35:25.069153 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.069128 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed272386-edb3-4e21-a30d-122218e304f9-kube-api-access-pj9mg" (OuterVolumeSpecName: "kube-api-access-pj9mg") pod "ed272386-edb3-4e21-a30d-122218e304f9" (UID: "ed272386-edb3-4e21-a30d-122218e304f9"). InnerVolumeSpecName "kube-api-access-pj9mg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:35:25.069538 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.069516 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ed272386-edb3-4e21-a30d-122218e304f9" (UID: "ed272386-edb3-4e21-a30d-122218e304f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:35:25.168275 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.168231 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-istio-envoy\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:35:25.168275 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.168267 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-workload-certs\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:35:25.168275 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.168281 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed272386-edb3-4e21-a30d-122218e304f9-istio-data\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:35:25.168895 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.168296 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed272386-edb3-4e21-a30d-122218e304f9-istio-podinfo\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:35:25.168895 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.168308 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pj9mg\" (UniqueName: \"kubernetes.io/projected/ed272386-edb3-4e21-a30d-122218e304f9-kube-api-access-pj9mg\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:35:25.168895 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.168321 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed272386-edb3-4e21-a30d-122218e304f9-istio-token\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:35:25.537467 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.537427 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr"] Jul 13 17:35:25.537916 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.537897 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed272386-edb3-4e21-a30d-122218e304f9" containerName="istio-proxy" Jul 13 17:35:25.538034 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.537918 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed272386-edb3-4e21-a30d-122218e304f9" containerName="istio-proxy" Jul 13 17:35:25.538034 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.537938 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed272386-edb3-4e21-a30d-122218e304f9" containerName="istio-proxy" Jul 13 17:35:25.538034 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.537947 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed272386-edb3-4e21-a30d-122218e304f9" containerName="istio-proxy" Jul 13 17:35:25.538034 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.538026 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed272386-edb3-4e21-a30d-122218e304f9" containerName="istio-proxy" Jul 13 17:35:25.540828 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.540792 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr" Jul 13 17:35:25.543961 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.543940 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k25z5\"" Jul 13 17:35:25.543961 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.543956 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 17:35:25.548323 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.548301 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr"] Jul 13 17:35:25.575413 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.575301 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feeb4543-063b-4e22-8521-23470b4c3592-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr\" (UID: \"feeb4543-063b-4e22-8521-23470b4c3592\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr" Jul 13 17:35:25.676154 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.676104 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feeb4543-063b-4e22-8521-23470b4c3592-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr\" (UID: \"feeb4543-063b-4e22-8521-23470b4c3592\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr" Jul 13 17:35:25.677128 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.677084 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feeb4543-063b-4e22-8521-23470b4c3592-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr\" (UID: \"feeb4543-063b-4e22-8521-23470b4c3592\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr" Jul 13 17:35:25.854480 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.854371 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr" Jul 13 17:35:25.896755 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.896223 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" event={"ID":"ed272386-edb3-4e21-a30d-122218e304f9","Type":"ContainerDied","Data":"5b60600f78acac988161e8a7fe52358b3a30aaf8bf0249ab9011aef71d05355d"} Jul 13 17:35:25.896755 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.896299 2564 scope.go:117] "RemoveContainer" containerID="8300bcb377d188d3efea246efd1d01a0d7dc9cba05a84b898a69977b0744c816" Jul 13 17:35:25.896755 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.896460 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9" Jul 13 17:35:25.928038 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.928005 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9"] Jul 13 17:35:25.932589 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:25.932560 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0e7283f7-openshift-default-7b894cd75-p8qv9"] Jul 13 17:35:26.042655 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:26.042624 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr"] Jul 13 17:35:26.044835 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:35:26.044800 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeeb4543_063b_4e22_8521_23470b4c3592.slice/crio-81b86048074cfae1cf0d25902bab2fa2baa5a45319e02c0d21598494c6234130 WatchSource:0}: Error finding container 81b86048074cfae1cf0d25902bab2fa2baa5a45319e02c0d21598494c6234130: Status 404 returned error can't find the container with id 81b86048074cfae1cf0d25902bab2fa2baa5a45319e02c0d21598494c6234130 Jul 13 17:35:26.903667 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:26.903629 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr" event={"ID":"feeb4543-063b-4e22-8521-23470b4c3592","Type":"ContainerStarted","Data":"0db0a77d009fdff6775e328a6be0c3ec9def5248af9bf6137c2266edcb13c421"} Jul 13 17:35:26.904090 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:26.903685 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr" event={"ID":"feeb4543-063b-4e22-8521-23470b4c3592","Type":"ContainerStarted","Data":"81b86048074cfae1cf0d25902bab2fa2baa5a45319e02c0d21598494c6234130"} Jul 13 17:35:26.921164 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:26.921109 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr" podStartSLOduration=1.9210846510000001 podStartE2EDuration="1.921084651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:35:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:35:26.918513606 +0000 UTC m=+1804.581288795" watchObservedRunningTime="2026-07-13 17:35:26.921084651 +0000 UTC m=+1804.583859851" Jul 13 17:35:26.960675 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:26.960639 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed272386-edb3-4e21-a30d-122218e304f9" path="/var/lib/kubelet/pods/ed272386-edb3-4e21-a30d-122218e304f9/volumes" Jul 13 17:35:27.910569 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:27.910535 2564 generic.go:356] "Generic (PLEG): container finished" podID="feeb4543-063b-4e22-8521-23470b4c3592" containerID="0db0a77d009fdff6775e328a6be0c3ec9def5248af9bf6137c2266edcb13c421" exitCode=7 Jul 13 17:35:27.911010 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:27.910622 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr" event={"ID":"feeb4543-063b-4e22-8521-23470b4c3592","Type":"ContainerDied","Data":"0db0a77d009fdff6775e328a6be0c3ec9def5248af9bf6137c2266edcb13c421"} Jul 13 17:35:27.911010 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:27.910833 2564 scope.go:117] "RemoveContainer" containerID="0db0a77d009fdff6775e328a6be0c3ec9def5248af9bf6137c2266edcb13c421" Jul 13 17:35:28.918803 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:28.918765 2564 generic.go:356] "Generic (PLEG): container finished" podID="feeb4543-063b-4e22-8521-23470b4c3592" containerID="3ce64b12fce42715f2c0f43c870e34837d9dd3d6e612254a8d56bd13a317f523" exitCode=7 Jul 13 17:35:28.919204 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:28.918896 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr" event={"ID":"feeb4543-063b-4e22-8521-23470b4c3592","Type":"ContainerDied","Data":"3ce64b12fce42715f2c0f43c870e34837d9dd3d6e612254a8d56bd13a317f523"} Jul 13 17:35:28.919204 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:28.918963 2564 scope.go:117] "RemoveContainer" containerID="0db0a77d009fdff6775e328a6be0c3ec9def5248af9bf6137c2266edcb13c421" Jul 13 17:35:28.919348 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:28.919222 2564 scope.go:117] "RemoveContainer" containerID="3ce64b12fce42715f2c0f43c870e34837d9dd3d6e612254a8d56bd13a317f523" Jul 13 17:35:28.919449 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:35:28.919427 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr_odh-ai-gateway-infra(feeb4543-063b-4e22-8521-23470b4c3592)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr" podUID="feeb4543-063b-4e22-8521-23470b4c3592" Jul 13 17:35:29.933860 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:29.933826 2564 scope.go:117] "RemoveContainer" containerID="3ce64b12fce42715f2c0f43c870e34837d9dd3d6e612254a8d56bd13a317f523" Jul 13 17:35:29.934377 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:35:29.934054 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr_odh-ai-gateway-infra(feeb4543-063b-4e22-8521-23470b4c3592)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr" podUID="feeb4543-063b-4e22-8521-23470b4c3592" Jul 13 17:35:31.032074 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:31.032042 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:31.097144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:31.097110 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:32.951349 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:32.951315 2564 generic.go:356] "Generic (PLEG): container finished" podID="df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" containerID="58f5536c50686a86c9cacea3bde0bc76915552218e2e4cf9d431fcadfb2f8425" exitCode=0 Jul 13 17:35:32.951874 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:32.951387 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" event={"ID":"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8","Type":"ContainerDied","Data":"58f5536c50686a86c9cacea3bde0bc76915552218e2e4cf9d431fcadfb2f8425"} Jul 13 17:35:32.951874 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:32.951800 2564 scope.go:117] "RemoveContainer" containerID="58f5536c50686a86c9cacea3bde0bc76915552218e2e4cf9d431fcadfb2f8425" Jul 13 17:35:32.952834 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:32.952808 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:35:32.952903 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:32.952871 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:35:32.952941 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:32.952905 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:35:33.958863 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:33.958828 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" event={"ID":"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8","Type":"ContainerStarted","Data":"65330920b71a0bfa5c537ff55e920608f91c90ea22c60df53d3ac235d6a1a8dc"} Jul 13 17:35:34.332858 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:34.332760 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:34.333982 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:34.333958 2564 patch_prober.go:28] interesting pod/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 13 17:35:34.334039 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:34.334019 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" podUID="df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 13 17:35:34.802527 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:34.802488 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:35.332153 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:35.332116 2564 patch_prober.go:28] interesting pod/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 13 17:35:35.332563 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:35.332176 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" podUID="df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 13 17:35:36.332048 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:36.332000 2564 patch_prober.go:28] interesting pod/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 13 17:35:36.332273 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:36.332069 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" podUID="df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 13 17:35:36.863466 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:36.863429 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:37.332587 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:37.332548 2564 patch_prober.go:28] interesting pod/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 13 17:35:37.333010 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:37.332608 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" podUID="df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 13 17:35:38.245161 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:38.245129 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:38.336331 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:38.336298 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:38.337478 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:38.337460 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:35:40.955659 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:40.955623 2564 scope.go:117] "RemoveContainer" containerID="3ce64b12fce42715f2c0f43c870e34837d9dd3d6e612254a8d56bd13a317f523" Jul 13 17:35:42.016942 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:42.016843 2564 generic.go:356] "Generic (PLEG): container finished" podID="feeb4543-063b-4e22-8521-23470b4c3592" containerID="dc7baf6cd96ed909d8daeb5e251c3e068a23b30244a77a0da4ea20cccc120814" exitCode=0 Jul 13 17:35:42.016942 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:42.016923 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr" event={"ID":"feeb4543-063b-4e22-8521-23470b4c3592","Type":"ContainerDied","Data":"dc7baf6cd96ed909d8daeb5e251c3e068a23b30244a77a0da4ea20cccc120814"} Jul 13 17:35:42.017445 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:42.016998 2564 scope.go:117] "RemoveContainer" containerID="3ce64b12fce42715f2c0f43c870e34837d9dd3d6e612254a8d56bd13a317f523" Jul 13 17:35:43.063233 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:43.063189 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr"] Jul 13 17:35:43.182492 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:43.182464 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr" Jul 13 17:35:43.262112 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:43.262066 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feeb4543-063b-4e22-8521-23470b4c3592-maas-api-service-ca\") pod \"feeb4543-063b-4e22-8521-23470b4c3592\" (UID: \"feeb4543-063b-4e22-8521-23470b4c3592\") " Jul 13 17:35:43.262766 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:43.262617 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feeb4543-063b-4e22-8521-23470b4c3592-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "feeb4543-063b-4e22-8521-23470b4c3592" (UID: "feeb4543-063b-4e22-8521-23470b4c3592"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:35:43.263210 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:43.263191 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feeb4543-063b-4e22-8521-23470b4c3592-maas-api-service-ca\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:35:44.032103 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:44.032068 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr" Jul 13 17:35:44.032310 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:44.032101 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr" event={"ID":"feeb4543-063b-4e22-8521-23470b4c3592","Type":"ContainerDied","Data":"81b86048074cfae1cf0d25902bab2fa2baa5a45319e02c0d21598494c6234130"} Jul 13 17:35:44.032310 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:44.032149 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81b86048074cfae1cf0d25902bab2fa2baa5a45319e02c0d21598494c6234130" Jul 13 17:35:44.062337 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:44.062299 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:44.075511 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:44.075474 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr"] Jul 13 17:35:44.085858 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:44.085830 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-cr9zr"] Jul 13 17:35:44.156705 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:44.156666 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:44.966670 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:44.966634 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="feeb4543-063b-4e22-8521-23470b4c3592" path="/var/lib/kubelet/pods/feeb4543-063b-4e22-8521-23470b4c3592/volumes" Jul 13 17:35:46.239604 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:46.239571 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:35:46.341406 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:35:46.341370 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:36:15.583363 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.583317 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-gjdks"] Jul 13 17:36:15.583884 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.583712 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feeb4543-063b-4e22-8521-23470b4c3592" containerName="revoke-keys" Jul 13 17:36:15.583884 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.583723 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="feeb4543-063b-4e22-8521-23470b4c3592" containerName="revoke-keys" Jul 13 17:36:15.583884 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.583750 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feeb4543-063b-4e22-8521-23470b4c3592" containerName="revoke-keys" Jul 13 17:36:15.583884 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.583755 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="feeb4543-063b-4e22-8521-23470b4c3592" containerName="revoke-keys" Jul 13 17:36:15.583884 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.583763 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feeb4543-063b-4e22-8521-23470b4c3592" containerName="revoke-keys" Jul 13 17:36:15.583884 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.583768 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="feeb4543-063b-4e22-8521-23470b4c3592" containerName="revoke-keys" Jul 13 17:36:15.583884 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.583832 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="feeb4543-063b-4e22-8521-23470b4c3592" containerName="revoke-keys" Jul 13 17:36:15.583884 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.583841 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="feeb4543-063b-4e22-8521-23470b4c3592" containerName="revoke-keys" Jul 13 17:36:15.583884 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.583849 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed272386-edb3-4e21-a30d-122218e304f9" containerName="istio-proxy" Jul 13 17:36:15.583884 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.583857 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="feeb4543-063b-4e22-8521-23470b4c3592" containerName="revoke-keys" Jul 13 17:36:15.586310 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.586293 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-gjdks" Jul 13 17:36:15.589281 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.589253 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k25z5\"" Jul 13 17:36:15.589389 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.589264 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 17:36:15.595158 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.595136 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-gjdks"] Jul 13 17:36:15.666289 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.666251 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60221c77-ad6d-4c47-b436-f9d84879e2be-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8e6203f1-gjdks\" (UID: \"60221c77-ad6d-4c47-b436-f9d84879e2be\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-gjdks" Jul 13 17:36:15.767367 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.767321 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60221c77-ad6d-4c47-b436-f9d84879e2be-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8e6203f1-gjdks\" (UID: \"60221c77-ad6d-4c47-b436-f9d84879e2be\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-gjdks" Jul 13 17:36:15.768009 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.767989 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60221c77-ad6d-4c47-b436-f9d84879e2be-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8e6203f1-gjdks\" (UID: \"60221c77-ad6d-4c47-b436-f9d84879e2be\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-gjdks" Jul 13 17:36:15.897817 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:15.897786 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-gjdks" Jul 13 17:36:16.031447 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:16.031422 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-gjdks"] Jul 13 17:36:16.033666 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:36:16.033640 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60221c77_ad6d_4c47_b436_f9d84879e2be.slice/crio-063b615bc3c3b675f34f6bfce522dbee235df0ac1ab01d62d4540205a86f9cc6 WatchSource:0}: Error finding container 063b615bc3c3b675f34f6bfce522dbee235df0ac1ab01d62d4540205a86f9cc6: Status 404 returned error can't find the container with id 063b615bc3c3b675f34f6bfce522dbee235df0ac1ab01d62d4540205a86f9cc6 Jul 13 17:36:16.209869 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:16.209781 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-gjdks" event={"ID":"60221c77-ad6d-4c47-b436-f9d84879e2be","Type":"ContainerStarted","Data":"063b615bc3c3b675f34f6bfce522dbee235df0ac1ab01d62d4540205a86f9cc6"} Jul 13 17:36:17.219984 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:17.219943 2564 generic.go:356] "Generic (PLEG): container finished" podID="60221c77-ad6d-4c47-b436-f9d84879e2be" containerID="a546b5d376f529ce57d684050db78eabc836f163cc24261e9ca8a712de1d6cc0" exitCode=0 Jul 13 17:36:17.220407 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:17.220016 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-gjdks" event={"ID":"60221c77-ad6d-4c47-b436-f9d84879e2be","Type":"ContainerDied","Data":"a546b5d376f529ce57d684050db78eabc836f163cc24261e9ca8a712de1d6cc0"} Jul 13 17:36:18.363316 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:18.363289 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-gjdks" Jul 13 17:36:18.493077 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:18.492965 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60221c77-ad6d-4c47-b436-f9d84879e2be-maas-api-service-ca\") pod \"60221c77-ad6d-4c47-b436-f9d84879e2be\" (UID: \"60221c77-ad6d-4c47-b436-f9d84879e2be\") " Jul 13 17:36:18.493368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:18.493342 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60221c77-ad6d-4c47-b436-f9d84879e2be-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "60221c77-ad6d-4c47-b436-f9d84879e2be" (UID: "60221c77-ad6d-4c47-b436-f9d84879e2be"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:36:18.593687 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:18.593651 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60221c77-ad6d-4c47-b436-f9d84879e2be-maas-api-service-ca\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:36:19.233186 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:19.233105 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-gjdks" Jul 13 17:36:19.233336 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:19.233104 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e6203f1-gjdks" event={"ID":"60221c77-ad6d-4c47-b436-f9d84879e2be","Type":"ContainerDied","Data":"063b615bc3c3b675f34f6bfce522dbee235df0ac1ab01d62d4540205a86f9cc6"} Jul 13 17:36:19.233336 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:19.233224 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="063b615bc3c3b675f34f6bfce522dbee235df0ac1ab01d62d4540205a86f9cc6" Jul 13 17:36:26.094895 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:26.094847 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:36:28.920662 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:28.920624 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:36:30.682668 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:30.682609 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:36:36.506242 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:36.506205 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:36:36.609875 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:36.609839 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:36:40.840819 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:40.840780 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s"] Jul 13 17:36:40.841348 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:40.841042 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" podUID="df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" containerName="istio-proxy" containerID="cri-o://65330920b71a0bfa5c537ff55e920608f91c90ea22c60df53d3ac235d6a1a8dc" gracePeriod=30 Jul 13 17:36:40.870231 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:40.870202 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:36:41.510490 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:41.510456 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:36:41.631882 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:41.631792 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:36:43.120912 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:43.120872 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:36:46.100037 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.100009 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:36:46.136805 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.136695 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-podinfo\") pod \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " Jul 13 17:36:46.136967 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.136834 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-credential-socket\") pod \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " Jul 13 17:36:46.136967 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.136876 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istiod-ca-cert\") pod \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " Jul 13 17:36:46.136967 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.136908 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-workload-certs\") pod \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " Jul 13 17:36:46.136967 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.136939 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-envoy\") pod \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " Jul 13 17:36:46.137189 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.136985 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbh9w\" (UniqueName: \"kubernetes.io/projected/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-kube-api-access-xbh9w\") pod \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " Jul 13 17:36:46.137189 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.137016 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-workload-socket\") pod \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " Jul 13 17:36:46.137189 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.137047 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-data\") pod \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " Jul 13 17:36:46.137189 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.137133 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" (UID: "df4ac151-c14a-42b2-bb39-c5d5f4c47ea8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:36:46.137189 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.137148 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-token\") pod \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\" (UID: \"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8\") " Jul 13 17:36:46.137504 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.137227 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" (UID: "df4ac151-c14a-42b2-bb39-c5d5f4c47ea8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:36:46.137504 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.137312 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" (UID: "df4ac151-c14a-42b2-bb39-c5d5f4c47ea8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:36:46.137504 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.137435 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" (UID: "df4ac151-c14a-42b2-bb39-c5d5f4c47ea8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:36:46.137671 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.137563 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-credential-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:36:46.137671 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.137584 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istiod-ca-cert\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:36:46.137671 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.137599 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-workload-certs\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:36:46.137671 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.137611 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-workload-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:36:46.138326 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.138303 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-data" (OuterVolumeSpecName: "istio-data") pod "df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" (UID: "df4ac151-c14a-42b2-bb39-c5d5f4c47ea8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:36:46.139621 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.139590 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-token" (OuterVolumeSpecName: "istio-token") pod "df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" (UID: "df4ac151-c14a-42b2-bb39-c5d5f4c47ea8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:36:46.139769 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.139642 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" (UID: "df4ac151-c14a-42b2-bb39-c5d5f4c47ea8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:36:46.139849 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.139823 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" (UID: "df4ac151-c14a-42b2-bb39-c5d5f4c47ea8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:36:46.139849 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.139830 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-kube-api-access-xbh9w" (OuterVolumeSpecName: "kube-api-access-xbh9w") pod "df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" (UID: "df4ac151-c14a-42b2-bb39-c5d5f4c47ea8"). InnerVolumeSpecName "kube-api-access-xbh9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:36:46.238246 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.238210 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-token\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:36:46.238246 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.238239 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-podinfo\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:36:46.238246 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.238247 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-envoy\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:36:46.238486 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.238256 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbh9w\" (UniqueName: \"kubernetes.io/projected/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-kube-api-access-xbh9w\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:36:46.238486 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.238265 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8-istio-data\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:36:46.383431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.383396 2564 generic.go:356] "Generic (PLEG): container finished" podID="df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" containerID="65330920b71a0bfa5c537ff55e920608f91c90ea22c60df53d3ac235d6a1a8dc" exitCode=0 Jul 13 17:36:46.383650 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.383454 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" Jul 13 17:36:46.383650 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.383493 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" event={"ID":"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8","Type":"ContainerDied","Data":"65330920b71a0bfa5c537ff55e920608f91c90ea22c60df53d3ac235d6a1a8dc"} Jul 13 17:36:46.383650 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.383564 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s" event={"ID":"df4ac151-c14a-42b2-bb39-c5d5f4c47ea8","Type":"ContainerDied","Data":"e9af95d452d43caeda4d5e91edc64d7b5de731221a0ca5da30c5c1a34f1d7906"} Jul 13 17:36:46.383650 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.383594 2564 scope.go:117] "RemoveContainer" containerID="65330920b71a0bfa5c537ff55e920608f91c90ea22c60df53d3ac235d6a1a8dc" Jul 13 17:36:46.393855 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.393838 2564 scope.go:117] "RemoveContainer" containerID="58f5536c50686a86c9cacea3bde0bc76915552218e2e4cf9d431fcadfb2f8425" Jul 13 17:36:46.402396 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.402378 2564 scope.go:117] "RemoveContainer" containerID="65330920b71a0bfa5c537ff55e920608f91c90ea22c60df53d3ac235d6a1a8dc" Jul 13 17:36:46.402663 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:36:46.402643 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65330920b71a0bfa5c537ff55e920608f91c90ea22c60df53d3ac235d6a1a8dc\": container with ID starting with 65330920b71a0bfa5c537ff55e920608f91c90ea22c60df53d3ac235d6a1a8dc not found: ID does not exist" containerID="65330920b71a0bfa5c537ff55e920608f91c90ea22c60df53d3ac235d6a1a8dc" Jul 13 17:36:46.402745 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.402673 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65330920b71a0bfa5c537ff55e920608f91c90ea22c60df53d3ac235d6a1a8dc"} err="failed to get container status \"65330920b71a0bfa5c537ff55e920608f91c90ea22c60df53d3ac235d6a1a8dc\": rpc error: code = NotFound desc = could not find container \"65330920b71a0bfa5c537ff55e920608f91c90ea22c60df53d3ac235d6a1a8dc\": container with ID starting with 65330920b71a0bfa5c537ff55e920608f91c90ea22c60df53d3ac235d6a1a8dc not found: ID does not exist" Jul 13 17:36:46.402745 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.402691 2564 scope.go:117] "RemoveContainer" containerID="58f5536c50686a86c9cacea3bde0bc76915552218e2e4cf9d431fcadfb2f8425" Jul 13 17:36:46.402967 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:36:46.402950 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58f5536c50686a86c9cacea3bde0bc76915552218e2e4cf9d431fcadfb2f8425\": container with ID starting with 58f5536c50686a86c9cacea3bde0bc76915552218e2e4cf9d431fcadfb2f8425 not found: ID does not exist" containerID="58f5536c50686a86c9cacea3bde0bc76915552218e2e4cf9d431fcadfb2f8425" Jul 13 17:36:46.403010 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.402974 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58f5536c50686a86c9cacea3bde0bc76915552218e2e4cf9d431fcadfb2f8425"} err="failed to get container status \"58f5536c50686a86c9cacea3bde0bc76915552218e2e4cf9d431fcadfb2f8425\": rpc error: code = NotFound desc = could not find container \"58f5536c50686a86c9cacea3bde0bc76915552218e2e4cf9d431fcadfb2f8425\": container with ID starting with 58f5536c50686a86c9cacea3bde0bc76915552218e2e4cf9d431fcadfb2f8425 not found: ID does not exist" Jul 13 17:36:46.408783 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.408761 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s"] Jul 13 17:36:46.412753 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.412718 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-8e6203f1-openshift-default-86847f5df5-gpm5s"] Jul 13 17:36:46.960456 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:46.960420 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" path="/var/lib/kubelet/pods/df4ac151-c14a-42b2-bb39-c5d5f4c47ea8/volumes" Jul 13 17:36:47.626461 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:47.626426 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:36:49.155523 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:49.155485 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:36:49.263678 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:49.263638 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:36:49.360342 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:49.360294 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:36:49.460375 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:49.460290 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:36:49.916442 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:36:49.916409 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:37:10.000830 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:37:10.000788 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:37:15.856479 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:37:15.856434 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:37:21.586598 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:37:21.586565 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:37:21.700094 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:37:21.700062 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:37:21.787794 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:37:21.787756 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:37:23.209708 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:37:23.209673 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:37:23.273131 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:37:23.273099 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:37:37.656527 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:37:37.656494 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:37:43.641891 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:37:43.641858 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:37:43.713921 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:37:43.713884 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:37:43.821580 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:37:43.821545 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:37:43.913656 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:37:43.913622 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:37:54.380958 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:37:54.380921 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:37:55.201856 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:37:55.201818 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:37:55.299256 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:37:55.299223 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:00.823547 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:00.823510 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:00.884628 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:00.884592 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:00.975651 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:00.975617 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:01.076141 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:01.076041 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:01.175995 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:01.175959 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:01.832661 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:01.832616 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:01.932546 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:01.932511 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:02.046410 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:02.046375 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:03.602645 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:03.602607 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:04.134661 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:04.134626 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:38:09.305802 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:09.305769 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:09.407828 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:09.407796 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:09.966423 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:09.966385 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:13.433623 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:13.433582 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:38:15.758690 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:15.758637 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:15.829906 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:15.829873 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:16.029931 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.029808 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:16.282993 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.282893 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2"] Jul 13 17:38:16.283367 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.283347 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" containerName="istio-proxy" Jul 13 17:38:16.283463 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.283369 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" containerName="istio-proxy" Jul 13 17:38:16.283463 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.283383 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60221c77-ad6d-4c47-b436-f9d84879e2be" containerName="revoke-keys" Jul 13 17:38:16.283463 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.283393 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="60221c77-ad6d-4c47-b436-f9d84879e2be" containerName="revoke-keys" Jul 13 17:38:16.283463 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.283412 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" containerName="istio-proxy" Jul 13 17:38:16.283463 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.283421 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" containerName="istio-proxy" Jul 13 17:38:16.283710 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.283527 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" containerName="istio-proxy" Jul 13 17:38:16.283710 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.283544 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="60221c77-ad6d-4c47-b436-f9d84879e2be" containerName="revoke-keys" Jul 13 17:38:16.287105 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.287081 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.290249 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.290221 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-b8ebfd6b-openshift-default-dockercfg-wn9sg\"" Jul 13 17:38:16.306313 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.306290 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2"] Jul 13 17:38:16.307548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.307525 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:16.352689 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.352655 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-token\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.352888 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.352702 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-data\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.352888 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.352788 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-credential-socket\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.352888 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.352824 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-workload-certs\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.352888 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.352843 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtwm5\" (UniqueName: \"kubernetes.io/projected/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-kube-api-access-dtwm5\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.353049 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.352895 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-envoy\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.353049 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.352916 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-workload-socket\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.353049 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.352933 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istiod-ca-cert\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.353049 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.352956 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-podinfo\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.453880 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.453840 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-envoy\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.454093 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.453887 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-workload-socket\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.454093 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.453914 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istiod-ca-cert\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.454093 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.453947 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-podinfo\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.454093 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.454004 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-token\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.454093 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.454051 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-data\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.454093 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.454081 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-credential-socket\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.454385 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.454217 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-workload-certs\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.454385 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.454251 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtwm5\" (UniqueName: \"kubernetes.io/projected/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-kube-api-access-dtwm5\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.454492 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.454404 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-workload-socket\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.454492 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.454479 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-data\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.454672 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.454649 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-workload-certs\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.454723 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.454707 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istiod-ca-cert\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.454803 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.454788 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-credential-socket\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.456898 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.456869 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-envoy\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.457013 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.456998 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-podinfo\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.463241 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.463218 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-token\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.463325 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.463242 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtwm5\" (UniqueName: \"kubernetes.io/projected/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-kube-api-access-dtwm5\") pod \"e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.597437 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.597347 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:16.745406 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.745375 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2"] Jul 13 17:38:16.746808 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:38:16.746778 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d3ed60a_693d_4aa8_abe3_8a797e2a4b13.slice/crio-1287ef7e98c273d110ca84981a665b04beb4114e14badec1c63f4823354fc29d WatchSource:0}: Error finding container 1287ef7e98c273d110ca84981a665b04beb4114e14badec1c63f4823354fc29d: Status 404 returned error can't find the container with id 1287ef7e98c273d110ca84981a665b04beb4114e14badec1c63f4823354fc29d Jul 13 17:38:16.749037 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.749005 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:38:16.749140 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.749068 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:38:16.749140 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.749104 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:38:16.873092 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.872980 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" event={"ID":"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13","Type":"ContainerStarted","Data":"c6b641b1416ce15ef190f4a40c33f043acd35e0505d3ba64ce2a9f234e31aa32"} Jul 13 17:38:16.873556 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.873098 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" event={"ID":"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13","Type":"ContainerStarted","Data":"1287ef7e98c273d110ca84981a665b04beb4114e14badec1c63f4823354fc29d"} Jul 13 17:38:16.875069 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.875039 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:16.901182 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:16.901126 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" podStartSLOduration=0.90110755 podStartE2EDuration="901.10755ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:38:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:38:16.898544474 +0000 UTC m=+1974.561319662" watchObservedRunningTime="2026-07-13 17:38:16.90110755 +0000 UTC m=+1974.563882814" Jul 13 17:38:17.598037 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:17.597998 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:17.599558 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:17.599531 2564 patch_prober.go:28] interesting pod/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 13 17:38:17.599692 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:17.599581 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" podUID="7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 13 17:38:18.601620 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:18.601590 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:18.884611 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:18.884534 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:18.885630 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:18.885609 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:18.975462 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:18.975425 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:19.065089 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:19.065049 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:19.139040 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:19.138948 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:19.672288 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:19.672254 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:19.778990 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:19.778956 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:23.430463 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:23.430428 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:24.078488 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:24.078449 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:24.177010 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:24.176964 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:24.488479 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:24.488445 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:24.932517 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:24.932484 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:38:30.497962 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:30.497926 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:30.598184 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:30.598147 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:30.860329 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:30.860227 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg"] Jul 13 17:38:30.860715 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:30.860699 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="df4ac151-c14a-42b2-bb39-c5d5f4c47ea8" containerName="istio-proxy" Jul 13 17:38:30.864042 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:30.864021 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg" Jul 13 17:38:30.867399 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:30.867373 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 17:38:30.867555 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:30.867414 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k25z5\"" Jul 13 17:38:30.873253 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:30.873216 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg"] Jul 13 17:38:30.996952 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:30.996909 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06511053-9573-478e-a627-56c78c0b876c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg\" (UID: \"06511053-9573-478e-a627-56c78c0b876c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg" Jul 13 17:38:31.097590 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:31.097552 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06511053-9573-478e-a627-56c78c0b876c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg\" (UID: \"06511053-9573-478e-a627-56c78c0b876c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg" Jul 13 17:38:31.098388 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:31.098361 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06511053-9573-478e-a627-56c78c0b876c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg\" (UID: \"06511053-9573-478e-a627-56c78c0b876c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg" Jul 13 17:38:31.178598 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:31.178562 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg" Jul 13 17:38:31.332856 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:31.332828 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg"] Jul 13 17:38:31.335333 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:38:31.335300 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06511053_9573_478e_a627_56c78c0b876c.slice/crio-1caea5a1f6d5a78dc25ea7e838fa072dd51e86b165bf30fb0bb7c4f9b06f065a WatchSource:0}: Error finding container 1caea5a1f6d5a78dc25ea7e838fa072dd51e86b165bf30fb0bb7c4f9b06f065a: Status 404 returned error can't find the container with id 1caea5a1f6d5a78dc25ea7e838fa072dd51e86b165bf30fb0bb7c4f9b06f065a Jul 13 17:38:31.957554 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:31.957471 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg" event={"ID":"06511053-9573-478e-a627-56c78c0b876c","Type":"ContainerStarted","Data":"926443fabfb0cb944b5f205939c639c115f370541b43e70f93738a8e447583f3"} Jul 13 17:38:31.957554 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:31.957515 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg" event={"ID":"06511053-9573-478e-a627-56c78c0b876c","Type":"ContainerStarted","Data":"1caea5a1f6d5a78dc25ea7e838fa072dd51e86b165bf30fb0bb7c4f9b06f065a"} Jul 13 17:38:31.976048 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:31.975996 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg" podStartSLOduration=1.975982078 podStartE2EDuration="1.975982078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:38:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:38:31.973522626 +0000 UTC m=+1989.636297814" watchObservedRunningTime="2026-07-13 17:38:31.975982078 +0000 UTC m=+1989.638757297" Jul 13 17:38:32.963800 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:32.963767 2564 generic.go:356] "Generic (PLEG): container finished" podID="06511053-9573-478e-a627-56c78c0b876c" containerID="926443fabfb0cb944b5f205939c639c115f370541b43e70f93738a8e447583f3" exitCode=0 Jul 13 17:38:32.964280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:32.963886 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg" event={"ID":"06511053-9573-478e-a627-56c78c0b876c","Type":"ContainerDied","Data":"926443fabfb0cb944b5f205939c639c115f370541b43e70f93738a8e447583f3"} Jul 13 17:38:32.965528 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:32.965491 2564 generic.go:356] "Generic (PLEG): container finished" podID="7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" containerID="c6b641b1416ce15ef190f4a40c33f043acd35e0505d3ba64ce2a9f234e31aa32" exitCode=0 Jul 13 17:38:32.965648 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:32.965534 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" event={"ID":"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13","Type":"ContainerDied","Data":"c6b641b1416ce15ef190f4a40c33f043acd35e0505d3ba64ce2a9f234e31aa32"} Jul 13 17:38:32.965963 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:32.965946 2564 scope.go:117] "RemoveContainer" containerID="c6b641b1416ce15ef190f4a40c33f043acd35e0505d3ba64ce2a9f234e31aa32" Jul 13 17:38:32.966935 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:32.966893 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:38:32.966990 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:32.966976 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:38:32.967030 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:32.967016 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:38:33.885250 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:33.885205 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:33.978524 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:33.978478 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" event={"ID":"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13","Type":"ContainerStarted","Data":"fc5d185fed697e8dcbc68689beef97e2f4a65be929de38b9db1c7b11a4e24beb"} Jul 13 17:38:34.147885 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:34.147858 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg" Jul 13 17:38:34.229256 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:34.229226 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06511053-9573-478e-a627-56c78c0b876c-maas-api-service-ca\") pod \"06511053-9573-478e-a627-56c78c0b876c\" (UID: \"06511053-9573-478e-a627-56c78c0b876c\") " Jul 13 17:38:34.229604 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:34.229580 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06511053-9573-478e-a627-56c78c0b876c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "06511053-9573-478e-a627-56c78c0b876c" (UID: "06511053-9573-478e-a627-56c78c0b876c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:38:34.330583 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:34.330539 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06511053-9573-478e-a627-56c78c0b876c-maas-api-service-ca\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:38:34.597792 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:34.597682 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:34.599020 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:34.598992 2564 patch_prober.go:28] interesting pod/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 13 17:38:34.599151 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:34.599045 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" podUID="7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 13 17:38:34.988699 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:34.988658 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg" Jul 13 17:38:34.989214 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:34.988776 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg" event={"ID":"06511053-9573-478e-a627-56c78c0b876c","Type":"ContainerDied","Data":"1caea5a1f6d5a78dc25ea7e838fa072dd51e86b165bf30fb0bb7c4f9b06f065a"} Jul 13 17:38:34.989214 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:34.988821 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1caea5a1f6d5a78dc25ea7e838fa072dd51e86b165bf30fb0bb7c4f9b06f065a" Jul 13 17:38:35.598611 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:35.598574 2564 patch_prober.go:28] interesting pod/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 13 17:38:35.598832 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:35.598628 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" podUID="7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 13 17:38:36.598475 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:36.598435 2564 patch_prober.go:28] interesting pod/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 13 17:38:36.598913 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:36.598494 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" podUID="7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 13 17:38:37.597974 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:37.597931 2564 patch_prober.go:28] interesting pod/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 13 17:38:37.598203 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:37.597994 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" podUID="7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 13 17:38:38.602480 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:38.602445 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:38.602931 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:38.602795 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:38.603425 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:38.603408 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:38:41.578963 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:41.578926 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:44.434118 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:44.434079 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:38:45.993878 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:45.993833 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:52.075755 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:52.075696 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:52.150959 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:52.150926 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:56.645498 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:56.645464 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2"] Jul 13 17:38:56.646055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:56.645742 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" podUID="7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" containerName="istio-proxy" containerID="cri-o://fc5d185fed697e8dcbc68689beef97e2f4a65be929de38b9db1c7b11a4e24beb" gracePeriod=30 Jul 13 17:38:56.664942 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:56.664911 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:56.892638 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:56.892600 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:57.028054 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:57.028008 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:57.116220 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:57.116184 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:38:57.244382 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:38:57.244343 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:39:01.890969 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.890943 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:39:01.980640 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.980541 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-workload-certs\") pod \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " Jul 13 17:39:01.980640 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.980584 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istiod-ca-cert\") pod \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " Jul 13 17:39:01.980640 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.980605 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-envoy\") pod \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " Jul 13 17:39:01.980640 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.980633 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-data\") pod \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " Jul 13 17:39:01.981012 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.980656 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-workload-socket\") pod \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " Jul 13 17:39:01.981012 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.980684 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-token\") pod \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " Jul 13 17:39:01.981012 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.980708 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-credential-socket\") pod \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " Jul 13 17:39:01.981012 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.980781 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtwm5\" (UniqueName: \"kubernetes.io/projected/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-kube-api-access-dtwm5\") pod \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " Jul 13 17:39:01.981012 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.980811 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-podinfo\") pod \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\" (UID: \"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13\") " Jul 13 17:39:01.981012 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.980981 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" (UID: "7d3ed60a-693d-4aa8-abe3-8a797e2a4b13"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:39:01.981340 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.981023 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" (UID: "7d3ed60a-693d-4aa8-abe3-8a797e2a4b13"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:39:01.981340 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.981101 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" (UID: "7d3ed60a-693d-4aa8-abe3-8a797e2a4b13"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:39:01.981340 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.981108 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-workload-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:39:01.981340 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.981192 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" (UID: "7d3ed60a-693d-4aa8-abe3-8a797e2a4b13"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:39:01.981340 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.981222 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-workload-certs\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:39:01.981791 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.981679 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-data" (OuterVolumeSpecName: "istio-data") pod "7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" (UID: "7d3ed60a-693d-4aa8-abe3-8a797e2a4b13"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:39:01.983091 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.983066 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" (UID: "7d3ed60a-693d-4aa8-abe3-8a797e2a4b13"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:39:01.983268 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.983244 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-kube-api-access-dtwm5" (OuterVolumeSpecName: "kube-api-access-dtwm5") pod "7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" (UID: "7d3ed60a-693d-4aa8-abe3-8a797e2a4b13"). InnerVolumeSpecName "kube-api-access-dtwm5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:39:01.983328 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.983249 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" (UID: "7d3ed60a-693d-4aa8-abe3-8a797e2a4b13"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:39:01.983328 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:01.983247 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-token" (OuterVolumeSpecName: "istio-token") pod "7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" (UID: "7d3ed60a-693d-4aa8-abe3-8a797e2a4b13"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:39:02.082252 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.082208 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-credential-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:39:02.082252 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.082247 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtwm5\" (UniqueName: \"kubernetes.io/projected/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-kube-api-access-dtwm5\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:39:02.082252 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.082257 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-podinfo\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:39:02.082495 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.082266 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istiod-ca-cert\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:39:02.082495 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.082275 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-envoy\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:39:02.082495 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.082282 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-data\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:39:02.082495 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.082291 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13-istio-token\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:39:02.139255 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.139217 2564 generic.go:356] "Generic (PLEG): container finished" podID="7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" containerID="fc5d185fed697e8dcbc68689beef97e2f4a65be929de38b9db1c7b11a4e24beb" exitCode=0 Jul 13 17:39:02.139426 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.139271 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" Jul 13 17:39:02.139426 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.139299 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" event={"ID":"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13","Type":"ContainerDied","Data":"fc5d185fed697e8dcbc68689beef97e2f4a65be929de38b9db1c7b11a4e24beb"} Jul 13 17:39:02.139426 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.139351 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2" event={"ID":"7d3ed60a-693d-4aa8-abe3-8a797e2a4b13","Type":"ContainerDied","Data":"1287ef7e98c273d110ca84981a665b04beb4114e14badec1c63f4823354fc29d"} Jul 13 17:39:02.139426 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.139374 2564 scope.go:117] "RemoveContainer" containerID="fc5d185fed697e8dcbc68689beef97e2f4a65be929de38b9db1c7b11a4e24beb" Jul 13 17:39:02.149130 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.149113 2564 scope.go:117] "RemoveContainer" containerID="c6b641b1416ce15ef190f4a40c33f043acd35e0505d3ba64ce2a9f234e31aa32" Jul 13 17:39:02.157617 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.157597 2564 scope.go:117] "RemoveContainer" containerID="fc5d185fed697e8dcbc68689beef97e2f4a65be929de38b9db1c7b11a4e24beb" Jul 13 17:39:02.157912 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:39:02.157893 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc5d185fed697e8dcbc68689beef97e2f4a65be929de38b9db1c7b11a4e24beb\": container with ID starting with fc5d185fed697e8dcbc68689beef97e2f4a65be929de38b9db1c7b11a4e24beb not found: ID does not exist" containerID="fc5d185fed697e8dcbc68689beef97e2f4a65be929de38b9db1c7b11a4e24beb" Jul 13 17:39:02.157977 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.157922 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc5d185fed697e8dcbc68689beef97e2f4a65be929de38b9db1c7b11a4e24beb"} err="failed to get container status \"fc5d185fed697e8dcbc68689beef97e2f4a65be929de38b9db1c7b11a4e24beb\": rpc error: code = NotFound desc = could not find container \"fc5d185fed697e8dcbc68689beef97e2f4a65be929de38b9db1c7b11a4e24beb\": container with ID starting with fc5d185fed697e8dcbc68689beef97e2f4a65be929de38b9db1c7b11a4e24beb not found: ID does not exist" Jul 13 17:39:02.157977 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.157942 2564 scope.go:117] "RemoveContainer" containerID="c6b641b1416ce15ef190f4a40c33f043acd35e0505d3ba64ce2a9f234e31aa32" Jul 13 17:39:02.158212 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:39:02.158195 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6b641b1416ce15ef190f4a40c33f043acd35e0505d3ba64ce2a9f234e31aa32\": container with ID starting with c6b641b1416ce15ef190f4a40c33f043acd35e0505d3ba64ce2a9f234e31aa32 not found: ID does not exist" containerID="c6b641b1416ce15ef190f4a40c33f043acd35e0505d3ba64ce2a9f234e31aa32" Jul 13 17:39:02.158267 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.158216 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6b641b1416ce15ef190f4a40c33f043acd35e0505d3ba64ce2a9f234e31aa32"} err="failed to get container status \"c6b641b1416ce15ef190f4a40c33f043acd35e0505d3ba64ce2a9f234e31aa32\": rpc error: code = NotFound desc = could not find container \"c6b641b1416ce15ef190f4a40c33f043acd35e0505d3ba64ce2a9f234e31aa32\": container with ID starting with c6b641b1416ce15ef190f4a40c33f043acd35e0505d3ba64ce2a9f234e31aa32 not found: ID does not exist" Jul 13 17:39:02.162923 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.162903 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2"] Jul 13 17:39:02.169069 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.169041 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-b8ebfd6b-openshift-default-5c5476ddf9-t7fm2"] Jul 13 17:39:02.961013 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:02.960966 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" path="/var/lib/kubelet/pods/7d3ed60a-693d-4aa8-abe3-8a797e2a4b13/volumes" Jul 13 17:39:03.550931 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:03.550893 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:39:03.689976 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:03.689942 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:39:03.824195 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:03.824111 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:39:03.999543 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:03.999507 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:39:10.168641 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:10.168599 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:39:10.254419 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:10.254386 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:39:17.032658 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.032623 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:39:17.495559 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.495520 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9"] Jul 13 17:39:17.496090 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.496073 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" containerName="istio-proxy" Jul 13 17:39:17.496189 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.496095 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" containerName="istio-proxy" Jul 13 17:39:17.496189 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.496116 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06511053-9573-478e-a627-56c78c0b876c" containerName="revoke-keys" Jul 13 17:39:17.496189 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.496125 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="06511053-9573-478e-a627-56c78c0b876c" containerName="revoke-keys" Jul 13 17:39:17.496285 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.496212 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" containerName="istio-proxy" Jul 13 17:39:17.496285 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.496224 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="06511053-9573-478e-a627-56c78c0b876c" containerName="revoke-keys" Jul 13 17:39:17.496285 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.496231 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" containerName="istio-proxy" Jul 13 17:39:17.499519 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.499496 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.502549 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.502525 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f4b51184-openshift-default-dockercfg-fctnh\"" Jul 13 17:39:17.524437 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.524410 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9"] Jul 13 17:39:17.536526 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.536498 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:39:17.624658 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.624615 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cfe407d-de63-419b-ba94-edae46043661-istio-podinfo\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.624884 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.624666 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cfe407d-de63-419b-ba94-edae46043661-istiod-ca-cert\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.624884 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.624687 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cfe407d-de63-419b-ba94-edae46043661-istio-token\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.624884 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.624710 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-workload-certs\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.624884 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.624836 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-credential-socket\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.624884 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.624874 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrv5m\" (UniqueName: \"kubernetes.io/projected/1cfe407d-de63-419b-ba94-edae46043661-kube-api-access-zrv5m\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.625107 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.624961 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-istio-data\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.625107 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.625030 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-istio-envoy\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.625107 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.625063 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-workload-socket\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.725990 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.725951 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-istio-data\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.726192 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.725996 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-istio-envoy\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.726192 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.726021 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-workload-socket\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.726192 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.726053 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cfe407d-de63-419b-ba94-edae46043661-istio-podinfo\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.726192 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.726089 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cfe407d-de63-419b-ba94-edae46043661-istiod-ca-cert\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.726192 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.726121 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cfe407d-de63-419b-ba94-edae46043661-istio-token\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.726192 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.726156 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-workload-certs\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.726494 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.726214 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-credential-socket\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.726494 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.726250 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrv5m\" (UniqueName: \"kubernetes.io/projected/1cfe407d-de63-419b-ba94-edae46043661-kube-api-access-zrv5m\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.726494 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.726482 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-istio-data\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.726645 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.726516 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-workload-socket\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.726716 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.726691 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-workload-certs\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.726805 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.726790 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-credential-socket\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.726949 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.726923 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cfe407d-de63-419b-ba94-edae46043661-istiod-ca-cert\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.729187 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.729158 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-istio-envoy\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.729300 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.729273 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cfe407d-de63-419b-ba94-edae46043661-istio-podinfo\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.740295 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.740256 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cfe407d-de63-419b-ba94-edae46043661-istio-token\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.740419 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.740373 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrv5m\" (UniqueName: \"kubernetes.io/projected/1cfe407d-de63-419b-ba94-edae46043661-kube-api-access-zrv5m\") pod \"e2e-mt-f4b51184-openshift-default-7964876879-dfnw9\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.772418 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.772330 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:39:17.809368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.809331 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:17.957349 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.957319 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9"] Jul 13 17:39:17.959001 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:39:17.958978 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfe407d_de63_419b_ba94_edae46043661.slice/crio-5cea2f2ba737731a15fac57870276d5db2261095bd630df41cbaeb6fa670e3ee WatchSource:0}: Error finding container 5cea2f2ba737731a15fac57870276d5db2261095bd630df41cbaeb6fa670e3ee: Status 404 returned error can't find the container with id 5cea2f2ba737731a15fac57870276d5db2261095bd630df41cbaeb6fa670e3ee Jul 13 17:39:17.961040 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.961011 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:39:17.961153 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.961094 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:39:17.961153 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:17.961140 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:39:18.228626 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:18.228584 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" event={"ID":"1cfe407d-de63-419b-ba94-edae46043661","Type":"ContainerStarted","Data":"e33d8867b9a9a18de3d877ada4eba7a55058a82ceebbb9937a22ad461e0aa8fc"} Jul 13 17:39:18.229144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:18.228649 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" event={"ID":"1cfe407d-de63-419b-ba94-edae46043661","Type":"ContainerStarted","Data":"5cea2f2ba737731a15fac57870276d5db2261095bd630df41cbaeb6fa670e3ee"} Jul 13 17:39:18.261498 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:18.261447 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" podStartSLOduration=1.2614310020000001 podStartE2EDuration="1.261431002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:39:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:39:18.259980561 +0000 UTC m=+2035.922755750" watchObservedRunningTime="2026-07-13 17:39:18.261431002 +0000 UTC m=+2035.924206193" Jul 13 17:39:18.809485 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:18.809448 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:19.815204 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:19.815170 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:20.239212 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:20.239178 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:20.240246 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:20.240223 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:20.416074 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:20.416026 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:39:20.560419 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:20.560333 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:39:20.667104 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:20.667070 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:39:20.928071 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:20.928028 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:39:21.266331 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:21.266220 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:39:21.358934 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:21.358903 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:39:23.106636 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:23.106602 2564 scope.go:117] "RemoveContainer" containerID="193374e44de0b75158eca61eb13a1d3ed7ee50999becebc46e4b54560a2c2cdb" Jul 13 17:39:24.728842 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:24.728805 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:39:26.140959 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:26.140918 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:39:29.156471 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:29.156437 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:39:34.329828 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:34.329741 2564 generic.go:356] "Generic (PLEG): container finished" podID="1cfe407d-de63-419b-ba94-edae46043661" containerID="e33d8867b9a9a18de3d877ada4eba7a55058a82ceebbb9937a22ad461e0aa8fc" exitCode=0 Jul 13 17:39:34.329828 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:34.329806 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" event={"ID":"1cfe407d-de63-419b-ba94-edae46043661","Type":"ContainerDied","Data":"e33d8867b9a9a18de3d877ada4eba7a55058a82ceebbb9937a22ad461e0aa8fc"} Jul 13 17:39:34.330270 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:34.330195 2564 scope.go:117] "RemoveContainer" containerID="e33d8867b9a9a18de3d877ada4eba7a55058a82ceebbb9937a22ad461e0aa8fc" Jul 13 17:39:34.331211 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:34.331186 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:39:34.331270 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:34.331256 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:39:34.331303 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:34.331293 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:39:35.240120 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:35.240078 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:35.337468 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:35.337430 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" event={"ID":"1cfe407d-de63-419b-ba94-edae46043661","Type":"ContainerStarted","Data":"fad0206e5f701814349ef9cbab8843c10cabb2543d5b5823ff627a34aa620770"} Jul 13 17:39:35.810176 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:35.810138 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:35.811187 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:35.811164 2564 patch_prober.go:28] interesting pod/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 13 17:39:35.811288 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:35.811232 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" podUID="1cfe407d-de63-419b-ba94-edae46043661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 13 17:39:36.810078 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:36.810035 2564 patch_prober.go:28] interesting pod/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 13 17:39:36.810496 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:36.810120 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" podUID="1cfe407d-de63-419b-ba94-edae46043661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 13 17:39:37.810115 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:37.810077 2564 patch_prober.go:28] interesting pod/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 13 17:39:37.810509 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:37.810138 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" podUID="1cfe407d-de63-419b-ba94-edae46043661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 13 17:39:38.810368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:38.810328 2564 patch_prober.go:28] interesting pod/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 13 17:39:38.810794 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:38.810394 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" podUID="1cfe407d-de63-419b-ba94-edae46043661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 13 17:39:39.814396 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:39.814358 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:39.814839 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:39.814642 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:39.815353 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:39.815337 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:39:42.435806 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:42.435768 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:39:52.244677 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:52.244637 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:39:52.861909 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:52.861870 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4b51184-f789r"] Jul 13 17:39:52.862233 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:52.862220 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" containerName="istio-proxy" Jul 13 17:39:52.862278 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:52.862235 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d3ed60a-693d-4aa8-abe3-8a797e2a4b13" containerName="istio-proxy" Jul 13 17:39:52.865318 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:52.865301 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4b51184-f789r" Jul 13 17:39:52.869896 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:52.869870 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k25z5\"" Jul 13 17:39:52.870141 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:52.870125 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 17:39:52.882672 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:52.882647 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4b51184-f789r"] Jul 13 17:39:52.944975 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:52.944943 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f949327-fe89-4565-9610-1d2a9c9d2b4a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f4b51184-f789r\" (UID: \"7f949327-fe89-4565-9610-1d2a9c9d2b4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4b51184-f789r" Jul 13 17:39:53.046407 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:53.046376 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f949327-fe89-4565-9610-1d2a9c9d2b4a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f4b51184-f789r\" (UID: \"7f949327-fe89-4565-9610-1d2a9c9d2b4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4b51184-f789r" Jul 13 17:39:53.047017 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:53.046991 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f949327-fe89-4565-9610-1d2a9c9d2b4a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f4b51184-f789r\" (UID: \"7f949327-fe89-4565-9610-1d2a9c9d2b4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4b51184-f789r" Jul 13 17:39:53.178639 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:53.178587 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4b51184-f789r" Jul 13 17:39:53.324894 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:53.324865 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4b51184-f789r"] Jul 13 17:39:53.326258 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:39:53.326230 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f949327_fe89_4565_9610_1d2a9c9d2b4a.slice/crio-87504bd5bddb9eca7b7e754077643abef08dde2b1f2babc7468254652ff72b05 WatchSource:0}: Error finding container 87504bd5bddb9eca7b7e754077643abef08dde2b1f2babc7468254652ff72b05: Status 404 returned error can't find the container with id 87504bd5bddb9eca7b7e754077643abef08dde2b1f2babc7468254652ff72b05 Jul 13 17:39:53.450748 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:53.450630 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4b51184-f789r" event={"ID":"7f949327-fe89-4565-9610-1d2a9c9d2b4a","Type":"ContainerStarted","Data":"87504bd5bddb9eca7b7e754077643abef08dde2b1f2babc7468254652ff72b05"} Jul 13 17:39:54.457816 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:54.457779 2564 generic.go:356] "Generic (PLEG): container finished" podID="7f949327-fe89-4565-9610-1d2a9c9d2b4a" containerID="95175445cfa8dc8b6897893a20959f4bc4091e6c2f238ecd2ccfef740210eead" exitCode=0 Jul 13 17:39:54.458199 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:54.457852 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4b51184-f789r" event={"ID":"7f949327-fe89-4565-9610-1d2a9c9d2b4a","Type":"ContainerDied","Data":"95175445cfa8dc8b6897893a20959f4bc4091e6c2f238ecd2ccfef740210eead"} Jul 13 17:39:55.594954 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:55.594925 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4b51184-f789r" Jul 13 17:39:55.667445 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:55.667403 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f949327-fe89-4565-9610-1d2a9c9d2b4a-maas-api-service-ca\") pod \"7f949327-fe89-4565-9610-1d2a9c9d2b4a\" (UID: \"7f949327-fe89-4565-9610-1d2a9c9d2b4a\") " Jul 13 17:39:55.667841 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:55.667818 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f949327-fe89-4565-9610-1d2a9c9d2b4a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7f949327-fe89-4565-9610-1d2a9c9d2b4a" (UID: "7f949327-fe89-4565-9610-1d2a9c9d2b4a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:39:55.769021 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:55.768935 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f949327-fe89-4565-9610-1d2a9c9d2b4a-maas-api-service-ca\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:39:56.470365 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:56.470323 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4b51184-f789r" event={"ID":"7f949327-fe89-4565-9610-1d2a9c9d2b4a","Type":"ContainerDied","Data":"87504bd5bddb9eca7b7e754077643abef08dde2b1f2babc7468254652ff72b05"} Jul 13 17:39:56.470550 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:56.470390 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87504bd5bddb9eca7b7e754077643abef08dde2b1f2babc7468254652ff72b05" Jul 13 17:39:56.470550 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:39:56.470336 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4b51184-f789r" Jul 13 17:40:03.880103 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:40:03.880067 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:40:07.648358 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:40:07.648324 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:40:07.942354 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:40:07.942264 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:40:13.023144 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:40:13.023108 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:40:23.135164 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:40:23.135129 2564 scope.go:117] "RemoveContainer" containerID="82166d52749cc3f0d559f072ddf90090d1cf1d6b2a212ecbd27fceffbd2d9d1a" Jul 13 17:40:45.031511 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:40:45.031473 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:40:45.132377 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:40:45.132343 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:40:50.109249 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:40:50.109216 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:40:50.435619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:40:50.435581 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:40:56.524362 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:40:56.524326 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:40:56.638168 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:40:56.638125 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:00.277885 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:00.277851 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9"] Jul 13 17:41:00.278335 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:00.278077 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" podUID="1cfe407d-de63-419b-ba94-edae46043661" containerName="istio-proxy" containerID="cri-o://fad0206e5f701814349ef9cbab8843c10cabb2543d5b5823ff627a34aa620770" gracePeriod=30 Jul 13 17:41:00.307116 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:00.307086 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:00.530183 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:00.530099 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:01.622087 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:01.622053 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:01.728608 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:01.728571 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:02.557062 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:02.557011 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:02.701371 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:02.701338 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:02.824638 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:02.824544 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:04.197211 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:04.197173 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:05.036877 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.036840 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:41:05.539126 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.539058 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:41:05.604786 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.604715 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-istio-envoy\") pod \"1cfe407d-de63-419b-ba94-edae46043661\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " Jul 13 17:41:05.604993 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.604796 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cfe407d-de63-419b-ba94-edae46043661-istio-podinfo\") pod \"1cfe407d-de63-419b-ba94-edae46043661\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " Jul 13 17:41:05.604993 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.604847 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-workload-certs\") pod \"1cfe407d-de63-419b-ba94-edae46043661\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " Jul 13 17:41:05.604993 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.604871 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-credential-socket\") pod \"1cfe407d-de63-419b-ba94-edae46043661\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " Jul 13 17:41:05.604993 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.604921 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrv5m\" (UniqueName: \"kubernetes.io/projected/1cfe407d-de63-419b-ba94-edae46043661-kube-api-access-zrv5m\") pod \"1cfe407d-de63-419b-ba94-edae46043661\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " Jul 13 17:41:05.604993 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.604952 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cfe407d-de63-419b-ba94-edae46043661-istiod-ca-cert\") pod \"1cfe407d-de63-419b-ba94-edae46043661\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " Jul 13 17:41:05.604993 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.604992 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-workload-socket\") pod \"1cfe407d-de63-419b-ba94-edae46043661\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " Jul 13 17:41:05.605331 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.605200 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1cfe407d-de63-419b-ba94-edae46043661" (UID: "1cfe407d-de63-419b-ba94-edae46043661"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:41:05.605331 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.605230 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cfe407d-de63-419b-ba94-edae46043661-istio-token\") pod \"1cfe407d-de63-419b-ba94-edae46043661\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " Jul 13 17:41:05.605331 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.605258 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1cfe407d-de63-419b-ba94-edae46043661" (UID: "1cfe407d-de63-419b-ba94-edae46043661"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:41:05.605331 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.605291 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-istio-data\") pod \"1cfe407d-de63-419b-ba94-edae46043661\" (UID: \"1cfe407d-de63-419b-ba94-edae46043661\") " Jul 13 17:41:05.605547 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.605424 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cfe407d-de63-419b-ba94-edae46043661-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1cfe407d-de63-419b-ba94-edae46043661" (UID: "1cfe407d-de63-419b-ba94-edae46043661"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:41:05.605547 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.605524 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1cfe407d-de63-419b-ba94-edae46043661" (UID: "1cfe407d-de63-419b-ba94-edae46043661"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:41:05.605660 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.605625 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-workload-certs\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:41:05.605660 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.605644 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-credential-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:41:05.605660 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.605657 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cfe407d-de63-419b-ba94-edae46043661-istiod-ca-cert\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:41:05.605870 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.605671 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-workload-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:41:05.606148 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.606125 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-istio-data" (OuterVolumeSpecName: "istio-data") pod "1cfe407d-de63-419b-ba94-edae46043661" (UID: "1cfe407d-de63-419b-ba94-edae46043661"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:41:05.607784 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.607755 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1cfe407d-de63-419b-ba94-edae46043661" (UID: "1cfe407d-de63-419b-ba94-edae46043661"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:41:05.607913 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.607776 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cfe407d-de63-419b-ba94-edae46043661-kube-api-access-zrv5m" (OuterVolumeSpecName: "kube-api-access-zrv5m") pod "1cfe407d-de63-419b-ba94-edae46043661" (UID: "1cfe407d-de63-419b-ba94-edae46043661"). InnerVolumeSpecName "kube-api-access-zrv5m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:41:05.607973 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.607957 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1cfe407d-de63-419b-ba94-edae46043661-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1cfe407d-de63-419b-ba94-edae46043661" (UID: "1cfe407d-de63-419b-ba94-edae46043661"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:41:05.608059 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.608036 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cfe407d-de63-419b-ba94-edae46043661-istio-token" (OuterVolumeSpecName: "istio-token") pod "1cfe407d-de63-419b-ba94-edae46043661" (UID: "1cfe407d-de63-419b-ba94-edae46043661"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:41:05.706699 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.706660 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrv5m\" (UniqueName: \"kubernetes.io/projected/1cfe407d-de63-419b-ba94-edae46043661-kube-api-access-zrv5m\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:41:05.706699 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.706693 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cfe407d-de63-419b-ba94-edae46043661-istio-token\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:41:05.706699 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.706703 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-istio-data\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:41:05.706699 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.706712 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cfe407d-de63-419b-ba94-edae46043661-istio-envoy\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:41:05.707001 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.706720 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cfe407d-de63-419b-ba94-edae46043661-istio-podinfo\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:41:05.864532 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.864445 2564 generic.go:356] "Generic (PLEG): container finished" podID="1cfe407d-de63-419b-ba94-edae46043661" containerID="fad0206e5f701814349ef9cbab8843c10cabb2543d5b5823ff627a34aa620770" exitCode=0 Jul 13 17:41:05.864687 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.864548 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" Jul 13 17:41:05.864687 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.864567 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" event={"ID":"1cfe407d-de63-419b-ba94-edae46043661","Type":"ContainerDied","Data":"fad0206e5f701814349ef9cbab8843c10cabb2543d5b5823ff627a34aa620770"} Jul 13 17:41:05.864687 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.864653 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9" event={"ID":"1cfe407d-de63-419b-ba94-edae46043661","Type":"ContainerDied","Data":"5cea2f2ba737731a15fac57870276d5db2261095bd630df41cbaeb6fa670e3ee"} Jul 13 17:41:05.864866 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.864699 2564 scope.go:117] "RemoveContainer" containerID="fad0206e5f701814349ef9cbab8843c10cabb2543d5b5823ff627a34aa620770" Jul 13 17:41:05.874982 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.874963 2564 scope.go:117] "RemoveContainer" containerID="e33d8867b9a9a18de3d877ada4eba7a55058a82ceebbb9937a22ad461e0aa8fc" Jul 13 17:41:05.883966 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.883948 2564 scope.go:117] "RemoveContainer" containerID="fad0206e5f701814349ef9cbab8843c10cabb2543d5b5823ff627a34aa620770" Jul 13 17:41:05.884224 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:41:05.884203 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fad0206e5f701814349ef9cbab8843c10cabb2543d5b5823ff627a34aa620770\": container with ID starting with fad0206e5f701814349ef9cbab8843c10cabb2543d5b5823ff627a34aa620770 not found: ID does not exist" containerID="fad0206e5f701814349ef9cbab8843c10cabb2543d5b5823ff627a34aa620770" Jul 13 17:41:05.884282 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.884234 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fad0206e5f701814349ef9cbab8843c10cabb2543d5b5823ff627a34aa620770"} err="failed to get container status \"fad0206e5f701814349ef9cbab8843c10cabb2543d5b5823ff627a34aa620770\": rpc error: code = NotFound desc = could not find container \"fad0206e5f701814349ef9cbab8843c10cabb2543d5b5823ff627a34aa620770\": container with ID starting with fad0206e5f701814349ef9cbab8843c10cabb2543d5b5823ff627a34aa620770 not found: ID does not exist" Jul 13 17:41:05.884282 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.884252 2564 scope.go:117] "RemoveContainer" containerID="e33d8867b9a9a18de3d877ada4eba7a55058a82ceebbb9937a22ad461e0aa8fc" Jul 13 17:41:05.884517 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:41:05.884454 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e33d8867b9a9a18de3d877ada4eba7a55058a82ceebbb9937a22ad461e0aa8fc\": container with ID starting with e33d8867b9a9a18de3d877ada4eba7a55058a82ceebbb9937a22ad461e0aa8fc not found: ID does not exist" containerID="e33d8867b9a9a18de3d877ada4eba7a55058a82ceebbb9937a22ad461e0aa8fc" Jul 13 17:41:05.884517 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.884471 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e33d8867b9a9a18de3d877ada4eba7a55058a82ceebbb9937a22ad461e0aa8fc"} err="failed to get container status \"e33d8867b9a9a18de3d877ada4eba7a55058a82ceebbb9937a22ad461e0aa8fc\": rpc error: code = NotFound desc = could not find container \"e33d8867b9a9a18de3d877ada4eba7a55058a82ceebbb9937a22ad461e0aa8fc\": container with ID starting with e33d8867b9a9a18de3d877ada4eba7a55058a82ceebbb9937a22ad461e0aa8fc not found: ID does not exist" Jul 13 17:41:05.896857 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.896827 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9"] Jul 13 17:41:05.899245 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:05.899225 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f4b51184-openshift-default-7964876879-dfnw9"] Jul 13 17:41:06.960801 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:06.960764 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cfe407d-de63-419b-ba94-edae46043661" path="/var/lib/kubelet/pods/1cfe407d-de63-419b-ba94-edae46043661/volumes" Jul 13 17:41:10.621074 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:10.621027 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:10.720191 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:10.720156 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:11.404993 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:11.404963 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:17.409375 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:17.409286 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:17.499105 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:17.499071 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:18.834645 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:18.834610 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:41:28.029585 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:28.029543 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:28.394161 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:28.394067 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:49.365922 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:49.365884 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:55.840615 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:55.840569 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:55.938577 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:55.938541 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:56.205293 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.205259 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:56.500574 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.500495 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx"] Jul 13 17:41:56.500896 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.500882 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f949327-fe89-4565-9610-1d2a9c9d2b4a" containerName="revoke-keys" Jul 13 17:41:56.500943 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.500898 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f949327-fe89-4565-9610-1d2a9c9d2b4a" containerName="revoke-keys" Jul 13 17:41:56.500943 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.500912 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cfe407d-de63-419b-ba94-edae46043661" containerName="istio-proxy" Jul 13 17:41:56.500943 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.500919 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cfe407d-de63-419b-ba94-edae46043661" containerName="istio-proxy" Jul 13 17:41:56.500943 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.500926 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cfe407d-de63-419b-ba94-edae46043661" containerName="istio-proxy" Jul 13 17:41:56.500943 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.500932 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cfe407d-de63-419b-ba94-edae46043661" containerName="istio-proxy" Jul 13 17:41:56.501105 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.500984 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f949327-fe89-4565-9610-1d2a9c9d2b4a" containerName="revoke-keys" Jul 13 17:41:56.501105 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.500994 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cfe407d-de63-419b-ba94-edae46043661" containerName="istio-proxy" Jul 13 17:41:56.501105 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.501000 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cfe407d-de63-419b-ba94-edae46043661" containerName="istio-proxy" Jul 13 17:41:56.503037 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.503022 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.507370 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.507350 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-6a3680-openshift-default-dockercfg-5bhnc\"" Jul 13 17:41:56.517605 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.517577 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx"] Jul 13 17:41:56.536171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.536141 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:56.611341 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.611291 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-credential-socket\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.611541 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.611347 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-workload-certs\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.611541 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.611413 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-envoy\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.611541 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.611459 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-data\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.611541 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.611497 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-token\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.611541 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.611526 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-podinfo\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.611800 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.611554 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-workload-socket\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.611800 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.611646 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z598\" (UniqueName: \"kubernetes.io/projected/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-kube-api-access-2z598\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.611800 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.611718 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istiod-ca-cert\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.713014 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.712974 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-credential-socket\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.713014 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.713019 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-workload-certs\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.713267 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.713040 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-envoy\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.713267 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.713175 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-data\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.713267 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.713218 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-token\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.713267 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.713241 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-podinfo\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.713267 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.713268 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-workload-socket\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.713501 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.713332 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z598\" (UniqueName: \"kubernetes.io/projected/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-kube-api-access-2z598\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.713501 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.713375 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istiod-ca-cert\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.713501 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.713471 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-credential-socket\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.713677 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.713656 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-data\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.713770 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.713687 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-workload-socket\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.713829 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.713786 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-workload-certs\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.714168 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.714141 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istiod-ca-cert\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.715687 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.715672 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-envoy\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.715939 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.715920 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-podinfo\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.725465 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.725443 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-token\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.725686 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.725662 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z598\" (UniqueName: \"kubernetes.io/projected/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-kube-api-access-2z598\") pod \"e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.813925 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.813825 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:56.973853 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.973830 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx"] Jul 13 17:41:56.975502 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:41:56.975472 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4c042a5_dac8_4b6f_844a_c1d12d10d7c7.slice/crio-c8506817e239814841e9f474b3fbea6b4b50fd84f22e918ec8de2b0d5954c514 WatchSource:0}: Error finding container c8506817e239814841e9f474b3fbea6b4b50fd84f22e918ec8de2b0d5954c514: Status 404 returned error can't find the container with id c8506817e239814841e9f474b3fbea6b4b50fd84f22e918ec8de2b0d5954c514 Jul 13 17:41:56.977914 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.977875 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:41:56.977995 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.977964 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:41:56.978034 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:56.978008 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:41:57.147869 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:57.147816 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:57.154170 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:57.154139 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" event={"ID":"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7","Type":"ContainerStarted","Data":"0a529a63265c22dda90e650a9b2b9830e6db8dc552d8b1258b914d50b9d258f7"} Jul 13 17:41:57.154331 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:57.154225 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" event={"ID":"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7","Type":"ContainerStarted","Data":"c8506817e239814841e9f474b3fbea6b4b50fd84f22e918ec8de2b0d5954c514"} Jul 13 17:41:57.182121 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:57.182071 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" podStartSLOduration=1.18205596 podStartE2EDuration="1.18205596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:41:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:41:57.180525582 +0000 UTC m=+2194.843300780" watchObservedRunningTime="2026-07-13 17:41:57.18205596 +0000 UTC m=+2194.844831148" Jul 13 17:41:57.814083 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:57.814043 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:58.819883 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:58.819851 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:59.087914 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:59.087804 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:59.165172 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:59.165136 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:59.166321 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:59.166285 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:41:59.183119 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:59.183086 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:59.346642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:59.346558 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:59.596941 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:59.596849 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:41:59.975064 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:41:59.975029 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:00.117518 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:00.117480 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:03.612385 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:03.612350 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:03.961958 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:03.961922 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:04.296792 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.296682 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq"] Jul 13 17:42:04.299643 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.299618 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.302549 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.302526 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-587790-openshift-default-dockercfg-trkpm\"" Jul 13 17:42:04.354100 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.354057 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:04.376819 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.376786 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq"] Jul 13 17:42:04.391882 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.391838 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-podinfo\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.392193 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.392167 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vldlh\" (UniqueName: \"kubernetes.io/projected/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-kube-api-access-vldlh\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.392358 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.392340 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istiod-ca-cert\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.392514 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.392500 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-data\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.392679 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.392665 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-credential-socket\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.392854 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.392835 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-workload-socket\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.393002 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.392987 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-workload-certs\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.393119 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.393104 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-envoy\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.393240 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.393229 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-token\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.491976 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.491941 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:04.494118 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.494086 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-workload-certs\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.494255 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.494123 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-envoy\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.494255 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.494166 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-token\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.494255 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.494204 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-podinfo\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.494255 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.494219 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vldlh\" (UniqueName: \"kubernetes.io/projected/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-kube-api-access-vldlh\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.494255 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.494237 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istiod-ca-cert\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.494486 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.494277 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-data\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.494486 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.494335 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-credential-socket\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.494486 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.494365 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-workload-socket\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.494649 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.494567 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-workload-certs\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.494882 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.494858 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-data\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.494972 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.494886 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-credential-socket\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.494972 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.494943 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-workload-socket\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.495132 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.495114 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istiod-ca-cert\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.497208 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.497188 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-envoy\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.497529 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.497510 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-podinfo\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.507539 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.507514 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-token\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.508033 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.508016 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vldlh\" (UniqueName: \"kubernetes.io/projected/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-kube-api-access-vldlh\") pod \"e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.610578 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.610480 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:04.762968 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.762937 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq"] Jul 13 17:42:04.764928 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:42:04.764899 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c1e7060_1f92_4bf4_a713_d92eb1dd61e2.slice/crio-5aa5142376c29386fa078a9652960e25f42fce13d87c3aab4d8002136ae86f85 WatchSource:0}: Error finding container 5aa5142376c29386fa078a9652960e25f42fce13d87c3aab4d8002136ae86f85: Status 404 returned error can't find the container with id 5aa5142376c29386fa078a9652960e25f42fce13d87c3aab4d8002136ae86f85 Jul 13 17:42:04.767174 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.767137 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:42:04.767266 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.767250 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:42:04.767308 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:04.767295 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:42:05.202146 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:05.202103 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" event={"ID":"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2","Type":"ContainerStarted","Data":"18f863d439d5ccd751b2a58490270b4ffcf6e80772364ef7aae72b114ee1efd1"} Jul 13 17:42:05.202349 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:05.202181 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" event={"ID":"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2","Type":"ContainerStarted","Data":"5aa5142376c29386fa078a9652960e25f42fce13d87c3aab4d8002136ae86f85"} Jul 13 17:42:05.231815 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:05.231751 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" podStartSLOduration=1.231706163 podStartE2EDuration="1.231706163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:42:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:42:05.22699007 +0000 UTC m=+2202.889765260" watchObservedRunningTime="2026-07-13 17:42:05.231706163 +0000 UTC m=+2202.894481352" Jul 13 17:42:05.611096 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:05.611003 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:05.615629 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:05.615602 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:06.207407 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:06.207372 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:06.208359 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:06.208336 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:07.271057 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:07.270954 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:07.292274 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:07.292237 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:42:07.387216 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:07.387177 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:07.491281 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:07.491243 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:07.587603 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:07.587522 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:08.026176 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:08.026146 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:08.149552 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:08.149511 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:11.733623 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:11.733582 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:14.904910 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:14.904089 2564 patch_prober.go:28] interesting pod/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": read tcp 10.133.0.2:59152->10.133.0.58:15021: read: connection reset by peer" start-of-body= Jul 13 17:42:14.904910 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:14.904170 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" podUID="e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": read tcp 10.133.0.2:59152->10.133.0.58:15021: read: connection reset by peer" Jul 13 17:42:15.261441 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:15.261354 2564 generic.go:356] "Generic (PLEG): container finished" podID="e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" containerID="0a529a63265c22dda90e650a9b2b9830e6db8dc552d8b1258b914d50b9d258f7" exitCode=0 Jul 13 17:42:15.261441 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:15.261416 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" event={"ID":"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7","Type":"ContainerDied","Data":"0a529a63265c22dda90e650a9b2b9830e6db8dc552d8b1258b914d50b9d258f7"} Jul 13 17:42:15.261814 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:15.261800 2564 scope.go:117] "RemoveContainer" containerID="0a529a63265c22dda90e650a9b2b9830e6db8dc552d8b1258b914d50b9d258f7" Jul 13 17:42:15.262901 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:15.262867 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:42:15.263010 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:15.262943 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:42:15.263010 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:15.262979 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:42:16.269274 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:16.269235 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" event={"ID":"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7","Type":"ContainerStarted","Data":"8943a6cff98eecf16fff3145043daee79ac20e323af0e591726bf6da7537df66"} Jul 13 17:42:16.814482 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:16.814432 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:42:16.814720 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:16.814615 2564 patch_prober.go:28] interesting pod/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 13 17:42:16.814720 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:16.814677 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" podUID="e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 13 17:42:17.149997 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:17.149965 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:17.814457 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:17.814415 2564 patch_prober.go:28] interesting pod/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 13 17:42:17.815110 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:17.815072 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" podUID="e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 13 17:42:18.815254 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:18.815216 2564 patch_prober.go:28] interesting pod/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 13 17:42:18.815890 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:18.815854 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" podUID="e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 13 17:42:19.814290 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:19.814250 2564 patch_prober.go:28] interesting pod/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 13 17:42:19.814651 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:19.814620 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" podUID="e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 13 17:42:20.817857 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:20.817829 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:42:20.818326 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:20.818303 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:42:20.819064 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:20.819044 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:42:20.910002 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:20.909965 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:21.644678 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:21.644640 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:22.249916 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:22.249877 2564 patch_prober.go:28] interesting pod/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 13 17:42:22.250333 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:22.249946 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" podUID="8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 13 17:42:22.344795 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:22.344760 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:42:23.171481 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:23.171443 2564 scope.go:117] "RemoveContainer" containerID="dc7baf6cd96ed909d8daeb5e251c3e068a23b30244a77a0da4ea20cccc120814" Jul 13 17:42:24.328077 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:24.328042 2564 generic.go:356] "Generic (PLEG): container finished" podID="8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" containerID="18f863d439d5ccd751b2a58490270b4ffcf6e80772364ef7aae72b114ee1efd1" exitCode=0 Jul 13 17:42:24.328498 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:24.328099 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" event={"ID":"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2","Type":"ContainerDied","Data":"18f863d439d5ccd751b2a58490270b4ffcf6e80772364ef7aae72b114ee1efd1"} Jul 13 17:42:24.328498 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:24.328479 2564 scope.go:117] "RemoveContainer" containerID="18f863d439d5ccd751b2a58490270b4ffcf6e80772364ef7aae72b114ee1efd1" Jul 13 17:42:24.329567 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:24.329542 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:42:24.329655 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:24.329641 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:42:24.329694 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:24.329677 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:42:25.334835 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:25.334801 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" event={"ID":"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2","Type":"ContainerStarted","Data":"67b04d197263a27b8e750e5f01f3960b9c585e0bd7ae801b66177b77de4c464d"} Jul 13 17:42:25.611222 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:25.611124 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:25.611383 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:25.611357 2564 patch_prober.go:28] interesting pod/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 13 17:42:25.611426 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:25.611408 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" podUID="8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 13 17:42:26.611108 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:26.611067 2564 patch_prober.go:28] interesting pod/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 13 17:42:26.611513 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:26.611126 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" podUID="8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 13 17:42:27.611416 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:27.611378 2564 patch_prober.go:28] interesting pod/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 13 17:42:27.611846 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:27.611447 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" podUID="8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 13 17:42:28.611091 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:28.611048 2564 patch_prober.go:28] interesting pod/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 13 17:42:28.611255 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:28.611107 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" podUID="8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 13 17:42:28.921783 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:28.921709 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:29.023592 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:29.023551 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:29.615092 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:29.615059 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:29.615475 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:29.615448 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:29.616537 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:29.616515 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:42:35.742140 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:35.742099 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:42:39.185487 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:39.185420 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:42:48.153653 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:42:48.153618 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:43:10.524102 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:10.524067 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:43:11.472225 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:11.472184 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:43:11.564821 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:11.564786 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg"] Jul 13 17:43:11.567648 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:11.567621 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg" Jul 13 17:43:11.571000 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:11.570973 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 17:43:11.571132 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:11.571030 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k25z5\"" Jul 13 17:43:11.578910 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:11.578887 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg"] Jul 13 17:43:11.605545 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:11.605513 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg\" (UID: \"c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg" Jul 13 17:43:11.707073 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:11.707030 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg\" (UID: \"c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg" Jul 13 17:43:11.707685 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:11.707662 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg\" (UID: \"c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg" Jul 13 17:43:11.880098 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:11.880010 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg" Jul 13 17:43:12.024086 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:12.024059 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg"] Jul 13 17:43:12.025623 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:43:12.025600 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d77ff2_c57b_48ff_a57b_cb8c1ccdaebe.slice/crio-e526ffb07c006bb7fca33ea0065c5c698e53d8d1e4947ca0d68f9e59c596c91c WatchSource:0}: Error finding container e526ffb07c006bb7fca33ea0065c5c698e53d8d1e4947ca0d68f9e59c596c91c: Status 404 returned error can't find the container with id e526ffb07c006bb7fca33ea0065c5c698e53d8d1e4947ca0d68f9e59c596c91c Jul 13 17:43:12.130086 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:12.130032 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:43:12.225112 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:12.225072 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:43:12.610224 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:12.610130 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg" event={"ID":"c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe","Type":"ContainerStarted","Data":"a0f59112a9ce51a4b25c6250d22cc3c2360abc7d636794165bec6c87c2fbcfd6"} Jul 13 17:43:12.610224 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:12.610183 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg" event={"ID":"c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe","Type":"ContainerStarted","Data":"e526ffb07c006bb7fca33ea0065c5c698e53d8d1e4947ca0d68f9e59c596c91c"} Jul 13 17:43:12.630771 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:12.630695 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg" podStartSLOduration=1.630678317 podStartE2EDuration="1.630678317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:43:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:43:12.629468028 +0000 UTC m=+2270.292243218" watchObservedRunningTime="2026-07-13 17:43:12.630678317 +0000 UTC m=+2270.293453506" Jul 13 17:43:13.617572 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:13.617531 2564 generic.go:356] "Generic (PLEG): container finished" podID="c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe" containerID="a0f59112a9ce51a4b25c6250d22cc3c2360abc7d636794165bec6c87c2fbcfd6" exitCode=0 Jul 13 17:43:13.618008 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:13.617578 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg" event={"ID":"c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe","Type":"ContainerDied","Data":"a0f59112a9ce51a4b25c6250d22cc3c2360abc7d636794165bec6c87c2fbcfd6"} Jul 13 17:43:14.760713 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:14.760688 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg" Jul 13 17:43:14.835491 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:14.835453 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe-maas-api-service-ca\") pod \"c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe\" (UID: \"c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe\") " Jul 13 17:43:14.835869 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:14.835848 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe" (UID: "c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:43:14.936963 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:14.936930 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe-maas-api-service-ca\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:43:15.629556 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:15.629525 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg" Jul 13 17:43:15.629556 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:15.629554 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg" event={"ID":"c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe","Type":"ContainerDied","Data":"e526ffb07c006bb7fca33ea0065c5c698e53d8d1e4947ca0d68f9e59c596c91c"} Jul 13 17:43:15.629881 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:15.629599 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e526ffb07c006bb7fca33ea0065c5c698e53d8d1e4947ca0d68f9e59c596c91c" Jul 13 17:43:16.234943 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:16.234906 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:43:22.084003 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:22.083970 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:43:26.695538 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:26.695497 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:43:32.136911 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:32.136877 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:43:33.452741 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:33.452704 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:43:33.553097 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:33.553052 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:43:37.476466 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:37.476423 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx"] Jul 13 17:43:37.476984 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:37.476670 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" podUID="e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" containerName="istio-proxy" containerID="cri-o://8943a6cff98eecf16fff3145043daee79ac20e323af0e591726bf6da7537df66" gracePeriod=30 Jul 13 17:43:37.557865 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:37.557830 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:43:37.679514 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:37.679472 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:43:38.420997 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:38.420964 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:43:38.521380 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:38.521342 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:43:42.731343 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.731314 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:43:42.785349 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.785252 2564 generic.go:356] "Generic (PLEG): container finished" podID="e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" containerID="8943a6cff98eecf16fff3145043daee79ac20e323af0e591726bf6da7537df66" exitCode=0 Jul 13 17:43:42.785349 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.785315 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" Jul 13 17:43:42.785550 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.785353 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" event={"ID":"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7","Type":"ContainerDied","Data":"8943a6cff98eecf16fff3145043daee79ac20e323af0e591726bf6da7537df66"} Jul 13 17:43:42.785550 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.785415 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx" event={"ID":"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7","Type":"ContainerDied","Data":"c8506817e239814841e9f474b3fbea6b4b50fd84f22e918ec8de2b0d5954c514"} Jul 13 17:43:42.785550 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.785439 2564 scope.go:117] "RemoveContainer" containerID="8943a6cff98eecf16fff3145043daee79ac20e323af0e591726bf6da7537df66" Jul 13 17:43:42.794955 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.794937 2564 scope.go:117] "RemoveContainer" containerID="0a529a63265c22dda90e650a9b2b9830e6db8dc552d8b1258b914d50b9d258f7" Jul 13 17:43:42.803434 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.803413 2564 scope.go:117] "RemoveContainer" containerID="8943a6cff98eecf16fff3145043daee79ac20e323af0e591726bf6da7537df66" Jul 13 17:43:42.803684 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:43:42.803664 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8943a6cff98eecf16fff3145043daee79ac20e323af0e591726bf6da7537df66\": container with ID starting with 8943a6cff98eecf16fff3145043daee79ac20e323af0e591726bf6da7537df66 not found: ID does not exist" containerID="8943a6cff98eecf16fff3145043daee79ac20e323af0e591726bf6da7537df66" Jul 13 17:43:42.803779 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.803692 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8943a6cff98eecf16fff3145043daee79ac20e323af0e591726bf6da7537df66"} err="failed to get container status \"8943a6cff98eecf16fff3145043daee79ac20e323af0e591726bf6da7537df66\": rpc error: code = NotFound desc = could not find container \"8943a6cff98eecf16fff3145043daee79ac20e323af0e591726bf6da7537df66\": container with ID starting with 8943a6cff98eecf16fff3145043daee79ac20e323af0e591726bf6da7537df66 not found: ID does not exist" Jul 13 17:43:42.803779 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.803708 2564 scope.go:117] "RemoveContainer" containerID="0a529a63265c22dda90e650a9b2b9830e6db8dc552d8b1258b914d50b9d258f7" Jul 13 17:43:42.803938 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:43:42.803921 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a529a63265c22dda90e650a9b2b9830e6db8dc552d8b1258b914d50b9d258f7\": container with ID starting with 0a529a63265c22dda90e650a9b2b9830e6db8dc552d8b1258b914d50b9d258f7 not found: ID does not exist" containerID="0a529a63265c22dda90e650a9b2b9830e6db8dc552d8b1258b914d50b9d258f7" Jul 13 17:43:42.803979 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.803942 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a529a63265c22dda90e650a9b2b9830e6db8dc552d8b1258b914d50b9d258f7"} err="failed to get container status \"0a529a63265c22dda90e650a9b2b9830e6db8dc552d8b1258b914d50b9d258f7\": rpc error: code = NotFound desc = could not find container \"0a529a63265c22dda90e650a9b2b9830e6db8dc552d8b1258b914d50b9d258f7\": container with ID starting with 0a529a63265c22dda90e650a9b2b9830e6db8dc552d8b1258b914d50b9d258f7 not found: ID does not exist" Jul 13 17:43:42.886445 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.886401 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-workload-socket\") pod \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " Jul 13 17:43:42.886445 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.886452 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-data\") pod \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " Jul 13 17:43:42.886705 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.886500 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-credential-socket\") pod \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " Jul 13 17:43:42.886705 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.886537 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-workload-certs\") pod \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " Jul 13 17:43:42.886705 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.886554 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2z598\" (UniqueName: \"kubernetes.io/projected/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-kube-api-access-2z598\") pod \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " Jul 13 17:43:42.886705 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.886592 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-token\") pod \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " Jul 13 17:43:42.886705 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.886631 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-podinfo\") pod \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " Jul 13 17:43:42.886705 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.886682 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-envoy\") pod \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " Jul 13 17:43:42.886705 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.886706 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istiod-ca-cert\") pod \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\" (UID: \"e4c042a5-dac8-4b6f-844a-c1d12d10d7c7\") " Jul 13 17:43:42.887056 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.886783 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" (UID: "e4c042a5-dac8-4b6f-844a-c1d12d10d7c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:43:42.887056 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.886905 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" (UID: "e4c042a5-dac8-4b6f-844a-c1d12d10d7c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:43:42.887056 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.886927 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" (UID: "e4c042a5-dac8-4b6f-844a-c1d12d10d7c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:43:42.887216 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.887097 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-credential-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:43:42.887216 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.887118 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-workload-certs\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:43:42.887216 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.887133 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-workload-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:43:42.887216 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.887207 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" (UID: "e4c042a5-dac8-4b6f-844a-c1d12d10d7c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:43:42.887413 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.887392 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-data" (OuterVolumeSpecName: "istio-data") pod "e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" (UID: "e4c042a5-dac8-4b6f-844a-c1d12d10d7c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:43:42.889513 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.889490 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" (UID: "e4c042a5-dac8-4b6f-844a-c1d12d10d7c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:43:42.889630 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.889515 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-token" (OuterVolumeSpecName: "istio-token") pod "e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" (UID: "e4c042a5-dac8-4b6f-844a-c1d12d10d7c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:43:42.889630 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.889597 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" (UID: "e4c042a5-dac8-4b6f-844a-c1d12d10d7c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:43:42.889630 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.889612 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-kube-api-access-2z598" (OuterVolumeSpecName: "kube-api-access-2z598") pod "e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" (UID: "e4c042a5-dac8-4b6f-844a-c1d12d10d7c7"). InnerVolumeSpecName "kube-api-access-2z598". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:43:42.987676 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.987642 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-token\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:43:42.987676 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.987670 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-podinfo\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:43:42.987888 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.987683 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-envoy\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:43:42.987888 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.987695 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istiod-ca-cert\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:43:42.987888 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.987706 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-istio-data\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:43:42.987888 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:42.987716 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2z598\" (UniqueName: \"kubernetes.io/projected/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7-kube-api-access-2z598\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:43:43.105422 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:43.105333 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx"] Jul 13 17:43:43.115294 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:43.115264 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-6a3680-openshift-default-5877d9bf77-txngx"] Jul 13 17:43:44.960767 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:44.960713 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" path="/var/lib/kubelet/pods/e4c042a5-dac8-4b6f-844a-c1d12d10d7c7/volumes" Jul 13 17:43:48.379650 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:48.379612 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:43:51.838569 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:51.838533 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:43:52.887255 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:52.887219 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:43:58.854584 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:58.854546 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:43:58.962066 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:43:58.962023 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:03.588647 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:03.588614 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq"] Jul 13 17:44:03.589084 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:03.588888 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" podUID="8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" containerName="istio-proxy" containerID="cri-o://67b04d197263a27b8e750e5f01f3960b9c585e0bd7ae801b66177b77de4c464d" gracePeriod=30 Jul 13 17:44:03.621779 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:03.621701 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:03.849039 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:03.848955 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:04.319156 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.319118 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:04.675966 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.675926 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h"] Jul 13 17:44:04.676354 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.676270 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe" containerName="revoke-keys" Jul 13 17:44:04.676354 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.676280 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe" containerName="revoke-keys" Jul 13 17:44:04.676354 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.676300 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" containerName="istio-proxy" Jul 13 17:44:04.676354 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.676305 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" containerName="istio-proxy" Jul 13 17:44:04.676354 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.676311 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" containerName="istio-proxy" Jul 13 17:44:04.676354 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.676318 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" containerName="istio-proxy" Jul 13 17:44:04.676549 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.676370 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" containerName="istio-proxy" Jul 13 17:44:04.676549 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.676380 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe" containerName="revoke-keys" Jul 13 17:44:04.678633 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.678611 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.685710 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.685414 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-edd1bf-openshift-default-dockercfg-dpp64\"" Jul 13 17:44:04.697723 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.697688 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h"] Jul 13 17:44:04.717771 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.717714 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:04.774714 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.774671 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-envoy\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.775135 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.775091 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd45853b-4390-44a7-bae7-1549f7d75dd1-istiod-ca-cert\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.775357 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.775333 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tms4\" (UniqueName: \"kubernetes.io/projected/dd45853b-4390-44a7-bae7-1549f7d75dd1-kube-api-access-4tms4\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.775580 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.775559 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-credential-socket\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.775794 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.775776 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-podinfo\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.775968 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.775943 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-workload-certs\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.776107 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.776091 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-workload-socket\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.776274 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.776256 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-data\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.776435 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.776418 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-token\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.878001 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.877953 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-data\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.878187 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.878013 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-token\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.878187 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.878049 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-envoy\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.878281 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.878188 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd45853b-4390-44a7-bae7-1549f7d75dd1-istiod-ca-cert\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.878281 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.878223 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tms4\" (UniqueName: \"kubernetes.io/projected/dd45853b-4390-44a7-bae7-1549f7d75dd1-kube-api-access-4tms4\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.878281 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.878272 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-credential-socket\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.878423 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.878318 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-podinfo\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.878423 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.878354 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-workload-certs\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.878423 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.878382 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-workload-socket\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.878423 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.878397 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-data\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.878709 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.878683 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-credential-socket\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.878824 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.878705 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-workload-certs\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.878824 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.878813 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-workload-socket\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.878947 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.878925 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd45853b-4390-44a7-bae7-1549f7d75dd1-istiod-ca-cert\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.880597 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.880577 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-envoy\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.880958 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.880938 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-podinfo\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.888500 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.888475 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-token\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.888612 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.888586 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tms4\" (UniqueName: \"kubernetes.io/projected/dd45853b-4390-44a7-bae7-1549f7d75dd1-kube-api-access-4tms4\") pod \"e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:04.989325 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:04.989236 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:05.117893 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:05.117862 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:05.140010 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:05.136512 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h"] Jul 13 17:44:05.142351 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:05.142316 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:44:05.142448 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:05.142392 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:44:05.142448 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:05.142423 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:44:05.215503 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:05.215465 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:05.338281 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:05.338188 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:05.915589 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:05.915552 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" event={"ID":"dd45853b-4390-44a7-bae7-1549f7d75dd1","Type":"ContainerStarted","Data":"a167ba91de22d937352c5d060a5896bb50b5a1ab9a2ebe93631f4b42442bb725"} Jul 13 17:44:05.916060 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:05.915604 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" event={"ID":"dd45853b-4390-44a7-bae7-1549f7d75dd1","Type":"ContainerStarted","Data":"35c12eef18a2a4c7b2c0b4caaba936f35166c5ccf8a532fd69ef62c5656b5331"} Jul 13 17:44:05.944770 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:05.944703 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" podStartSLOduration=1.9446859239999998 podStartE2EDuration="1.944685924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:44:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:44:05.942284552 +0000 UTC m=+2323.605059742" watchObservedRunningTime="2026-07-13 17:44:05.944685924 +0000 UTC m=+2323.607461112" Jul 13 17:44:05.990256 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:05.990223 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:06.994978 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:06.994946 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:07.934244 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:07.934209 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:07.935313 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:07.935289 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:08.043242 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.043205 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:08.180538 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.180229 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:08.306536 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.306441 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:08.847138 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.847111 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:08.853214 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.853009 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:44:08.915412 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.915374 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-podinfo\") pod \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " Jul 13 17:44:08.915412 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.915412 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-workload-socket\") pod \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " Jul 13 17:44:08.915681 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.915480 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-token\") pod \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " Jul 13 17:44:08.915681 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.915500 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-credential-socket\") pod \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " Jul 13 17:44:08.915681 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.915522 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istiod-ca-cert\") pod \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " Jul 13 17:44:08.915681 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.915564 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vldlh\" (UniqueName: \"kubernetes.io/projected/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-kube-api-access-vldlh\") pod \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " Jul 13 17:44:08.915681 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.915592 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-data\") pod \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " Jul 13 17:44:08.915681 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.915629 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-workload-certs\") pod \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " Jul 13 17:44:08.915681 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.915654 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-envoy\") pod \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\" (UID: \"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2\") " Jul 13 17:44:08.916126 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.915827 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" (UID: "8c1e7060-1f92-4bf4-a713-d92eb1dd61e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:44:08.916126 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.915844 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" (UID: "8c1e7060-1f92-4bf4-a713-d92eb1dd61e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:44:08.916126 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.916021 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-credential-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:44:08.916126 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.916044 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-workload-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:44:08.916126 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.916108 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" (UID: "8c1e7060-1f92-4bf4-a713-d92eb1dd61e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:44:08.916538 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.916268 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" (UID: "8c1e7060-1f92-4bf4-a713-d92eb1dd61e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:44:08.916762 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.916717 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" (UID: "8c1e7060-1f92-4bf4-a713-d92eb1dd61e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:44:08.917995 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.917966 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" (UID: "8c1e7060-1f92-4bf4-a713-d92eb1dd61e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:44:08.918251 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.918236 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" (UID: "8c1e7060-1f92-4bf4-a713-d92eb1dd61e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:44:08.918409 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.918389 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" (UID: "8c1e7060-1f92-4bf4-a713-d92eb1dd61e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:44:08.918449 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.918404 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-kube-api-access-vldlh" (OuterVolumeSpecName: "kube-api-access-vldlh") pod "8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" (UID: "8c1e7060-1f92-4bf4-a713-d92eb1dd61e2"). InnerVolumeSpecName "kube-api-access-vldlh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:44:08.941140 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.941108 2564 generic.go:356] "Generic (PLEG): container finished" podID="8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" containerID="67b04d197263a27b8e750e5f01f3960b9c585e0bd7ae801b66177b77de4c464d" exitCode=0 Jul 13 17:44:08.941140 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.941156 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" Jul 13 17:44:08.941140 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.941193 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" event={"ID":"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2","Type":"ContainerDied","Data":"67b04d197263a27b8e750e5f01f3960b9c585e0bd7ae801b66177b77de4c464d"} Jul 13 17:44:08.941476 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.941253 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq" event={"ID":"8c1e7060-1f92-4bf4-a713-d92eb1dd61e2","Type":"ContainerDied","Data":"5aa5142376c29386fa078a9652960e25f42fce13d87c3aab4d8002136ae86f85"} Jul 13 17:44:08.941476 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.941280 2564 scope.go:117] "RemoveContainer" containerID="67b04d197263a27b8e750e5f01f3960b9c585e0bd7ae801b66177b77de4c464d" Jul 13 17:44:08.942321 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.942271 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:08.951311 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.951295 2564 scope.go:117] "RemoveContainer" containerID="18f863d439d5ccd751b2a58490270b4ffcf6e80772364ef7aae72b114ee1efd1" Jul 13 17:44:08.960658 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.960641 2564 scope.go:117] "RemoveContainer" containerID="67b04d197263a27b8e750e5f01f3960b9c585e0bd7ae801b66177b77de4c464d" Jul 13 17:44:08.960953 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:44:08.960934 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67b04d197263a27b8e750e5f01f3960b9c585e0bd7ae801b66177b77de4c464d\": container with ID starting with 67b04d197263a27b8e750e5f01f3960b9c585e0bd7ae801b66177b77de4c464d not found: ID does not exist" containerID="67b04d197263a27b8e750e5f01f3960b9c585e0bd7ae801b66177b77de4c464d" Jul 13 17:44:08.961050 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.960965 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67b04d197263a27b8e750e5f01f3960b9c585e0bd7ae801b66177b77de4c464d"} err="failed to get container status \"67b04d197263a27b8e750e5f01f3960b9c585e0bd7ae801b66177b77de4c464d\": rpc error: code = NotFound desc = could not find container \"67b04d197263a27b8e750e5f01f3960b9c585e0bd7ae801b66177b77de4c464d\": container with ID starting with 67b04d197263a27b8e750e5f01f3960b9c585e0bd7ae801b66177b77de4c464d not found: ID does not exist" Jul 13 17:44:08.961050 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.960991 2564 scope.go:117] "RemoveContainer" containerID="18f863d439d5ccd751b2a58490270b4ffcf6e80772364ef7aae72b114ee1efd1" Jul 13 17:44:08.961306 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:44:08.961287 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18f863d439d5ccd751b2a58490270b4ffcf6e80772364ef7aae72b114ee1efd1\": container with ID starting with 18f863d439d5ccd751b2a58490270b4ffcf6e80772364ef7aae72b114ee1efd1 not found: ID does not exist" containerID="18f863d439d5ccd751b2a58490270b4ffcf6e80772364ef7aae72b114ee1efd1" Jul 13 17:44:08.961365 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.961316 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18f863d439d5ccd751b2a58490270b4ffcf6e80772364ef7aae72b114ee1efd1"} err="failed to get container status \"18f863d439d5ccd751b2a58490270b4ffcf6e80772364ef7aae72b114ee1efd1\": rpc error: code = NotFound desc = could not find container \"18f863d439d5ccd751b2a58490270b4ffcf6e80772364ef7aae72b114ee1efd1\": container with ID starting with 18f863d439d5ccd751b2a58490270b4ffcf6e80772364ef7aae72b114ee1efd1 not found: ID does not exist" Jul 13 17:44:08.971325 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.971291 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq"] Jul 13 17:44:08.978525 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:08.978496 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-587790-openshift-default-7b8fbbcf-9l8gq"] Jul 13 17:44:09.017055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:09.017015 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-token\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:44:09.017055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:09.017054 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istiod-ca-cert\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:44:09.017314 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:09.017075 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vldlh\" (UniqueName: \"kubernetes.io/projected/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-kube-api-access-vldlh\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:44:09.017314 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:09.017089 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-data\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:44:09.017314 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:09.017104 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-workload-certs\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:44:09.017314 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:09.017116 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-envoy\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:44:09.017314 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:09.017127 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2-istio-podinfo\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:44:10.960609 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:10.960572 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" path="/var/lib/kubelet/pods/8c1e7060-1f92-4bf4-a713-d92eb1dd61e2/volumes" Jul 13 17:44:12.440637 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:12.440604 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:12.887778 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:12.887677 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:13.188830 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.188798 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs"] Jul 13 17:44:13.189263 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.189245 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" containerName="istio-proxy" Jul 13 17:44:13.189377 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.189265 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" containerName="istio-proxy" Jul 13 17:44:13.189377 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.189277 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" containerName="istio-proxy" Jul 13 17:44:13.189377 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.189285 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" containerName="istio-proxy" Jul 13 17:44:13.189534 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.189375 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" containerName="istio-proxy" Jul 13 17:44:13.189534 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.189392 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4c042a5-dac8-4b6f-844a-c1d12d10d7c7" containerName="istio-proxy" Jul 13 17:44:13.192532 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.192505 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.195534 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.195513 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-e386c8-openshift-default-dockercfg-dflvt\"" Jul 13 17:44:13.204851 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.204826 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs"] Jul 13 17:44:13.224434 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.224394 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:13.255017 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.254984 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrjnw\" (UniqueName: \"kubernetes.io/projected/81f574a6-06f5-406b-b8a3-436d157dd1e1-kube-api-access-qrjnw\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.255160 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.255032 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-credential-socket\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.255160 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.255051 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-workload-certs\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.255160 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.255082 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-data\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.255160 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.255099 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-podinfo\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.255160 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.255153 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-envoy\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.255368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.255184 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-token\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.255368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.255197 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81f574a6-06f5-406b-b8a3-436d157dd1e1-istiod-ca-cert\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.255368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.255221 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-workload-socket\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.328168 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.328098 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:13.356577 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.356546 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-token\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.356577 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.356579 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81f574a6-06f5-406b-b8a3-436d157dd1e1-istiod-ca-cert\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.356852 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.356606 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-workload-socket\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.356852 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.356653 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrjnw\" (UniqueName: \"kubernetes.io/projected/81f574a6-06f5-406b-b8a3-436d157dd1e1-kube-api-access-qrjnw\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.356852 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.356690 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-credential-socket\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.356852 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.356713 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-workload-certs\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.356852 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.356770 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-data\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.356852 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.356797 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-podinfo\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.356852 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.356832 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-envoy\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.357258 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.357205 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-data\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.357308 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.357282 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-workload-certs\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.357349 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.357304 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-workload-socket\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.357390 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.357359 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-credential-socket\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.357526 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.357504 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81f574a6-06f5-406b-b8a3-436d157dd1e1-istiod-ca-cert\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.359640 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.359581 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-podinfo\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.359801 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.359776 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-envoy\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.367556 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.367528 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-token\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.367805 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.367616 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrjnw\" (UniqueName: \"kubernetes.io/projected/81f574a6-06f5-406b-b8a3-436d157dd1e1-kube-api-access-qrjnw\") pod \"e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.503581 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.503474 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:13.568967 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.568827 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:44:13.662413 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.662389 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs"] Jul 13 17:44:13.664328 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:44:13.664301 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81f574a6_06f5_406b_b8a3_436d157dd1e1.slice/crio-463ca466962d14e27a234eebc65e6047e152fa02db2722d8dde168ca9c3ec231 WatchSource:0}: Error finding container 463ca466962d14e27a234eebc65e6047e152fa02db2722d8dde168ca9c3ec231: Status 404 returned error can't find the container with id 463ca466962d14e27a234eebc65e6047e152fa02db2722d8dde168ca9c3ec231 Jul 13 17:44:13.666641 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.666604 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:44:13.666781 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.666690 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:44:13.666781 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.666761 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:44:13.977256 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.977203 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" event={"ID":"81f574a6-06f5-406b-b8a3-436d157dd1e1","Type":"ContainerStarted","Data":"f53a68f8665d32819ebb9ea605cf4ac28b9ccb48f2820769f42a5168aa579714"} Jul 13 17:44:13.977474 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:13.977266 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" event={"ID":"81f574a6-06f5-406b-b8a3-436d157dd1e1","Type":"ContainerStarted","Data":"463ca466962d14e27a234eebc65e6047e152fa02db2722d8dde168ca9c3ec231"} Jul 13 17:44:14.007911 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:14.007858 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" podStartSLOduration=1.007838759 podStartE2EDuration="1.007838759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:44:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:44:14.003787775 +0000 UTC m=+2331.666562965" watchObservedRunningTime="2026-07-13 17:44:14.007838759 +0000 UTC m=+2331.670613947" Jul 13 17:44:14.503826 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:14.503795 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:14.508932 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:14.508904 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:14.986988 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:14.986956 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:14.988715 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:14.988689 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:16.003456 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:16.003418 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:16.157059 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:16.157021 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:16.290281 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:16.290182 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:16.848230 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:16.848197 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:16.959341 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:16.959300 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:20.499488 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:20.499434 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:22.035846 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:22.035721 2564 generic.go:356] "Generic (PLEG): container finished" podID="dd45853b-4390-44a7-bae7-1549f7d75dd1" containerID="a167ba91de22d937352c5d060a5896bb50b5a1ab9a2ebe93631f4b42442bb725" exitCode=0 Jul 13 17:44:22.035846 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:22.035797 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" event={"ID":"dd45853b-4390-44a7-bae7-1549f7d75dd1","Type":"ContainerDied","Data":"a167ba91de22d937352c5d060a5896bb50b5a1ab9a2ebe93631f4b42442bb725"} Jul 13 17:44:22.036448 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:22.036333 2564 scope.go:117] "RemoveContainer" containerID="a167ba91de22d937352c5d060a5896bb50b5a1ab9a2ebe93631f4b42442bb725" Jul 13 17:44:22.037439 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:22.037411 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:44:22.037538 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:22.037496 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:44:22.037599 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:22.037546 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:44:22.934630 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:22.934590 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:23.049102 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:23.049057 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" event={"ID":"dd45853b-4390-44a7-bae7-1549f7d75dd1","Type":"ContainerStarted","Data":"bd3df9a44555dff4f32e520e09bb44148a115b1020a8ab3998df39f625303d99"} Jul 13 17:44:23.994077 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:23.994041 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:23.994493 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:23.994469 2564 patch_prober.go:28] interesting pod/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 13 17:44:23.994666 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:23.994643 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" podUID="dd45853b-4390-44a7-bae7-1549f7d75dd1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 13 17:44:24.990666 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:24.990615 2564 patch_prober.go:28] interesting pod/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 13 17:44:24.991204 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:24.990689 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" podUID="dd45853b-4390-44a7-bae7-1549f7d75dd1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 13 17:44:25.990357 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:25.990307 2564 patch_prober.go:28] interesting pod/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 13 17:44:25.990593 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:25.990384 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" podUID="dd45853b-4390-44a7-bae7-1549f7d75dd1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 13 17:44:26.990158 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:26.990113 2564 patch_prober.go:28] interesting pod/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 13 17:44:26.990659 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:26.990183 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" podUID="dd45853b-4390-44a7-bae7-1549f7d75dd1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 13 17:44:27.994041 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:27.994002 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:27.994497 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:27.994292 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:27.994942 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:27.994923 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:29.036552 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:29.036520 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:29.091151 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:29.091065 2564 generic.go:356] "Generic (PLEG): container finished" podID="81f574a6-06f5-406b-b8a3-436d157dd1e1" containerID="f53a68f8665d32819ebb9ea605cf4ac28b9ccb48f2820769f42a5168aa579714" exitCode=0 Jul 13 17:44:29.091151 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:29.091138 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" event={"ID":"81f574a6-06f5-406b-b8a3-436d157dd1e1","Type":"ContainerDied","Data":"f53a68f8665d32819ebb9ea605cf4ac28b9ccb48f2820769f42a5168aa579714"} Jul 13 17:44:29.091550 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:29.091536 2564 scope.go:117] "RemoveContainer" containerID="f53a68f8665d32819ebb9ea605cf4ac28b9ccb48f2820769f42a5168aa579714" Jul 13 17:44:29.092541 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:29.092514 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:44:29.092604 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:29.092589 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:44:29.092639 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:29.092619 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:44:29.120946 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:29.120896 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:29.991890 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:29.991832 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:30.105778 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:30.105718 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" event={"ID":"81f574a6-06f5-406b-b8a3-436d157dd1e1","Type":"ContainerStarted","Data":"f8b9bd6eb2b1ef20d3855797f110a3ffd0c71e5eaafd451469455eb33b434848"} Jul 13 17:44:30.504324 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:30.504281 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:30.505621 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:30.505594 2564 patch_prober.go:28] interesting pod/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 13 17:44:30.505755 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:30.505651 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" podUID="81f574a6-06f5-406b-b8a3-436d157dd1e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 13 17:44:31.338971 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:31.338936 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:44:31.504049 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:31.504007 2564 patch_prober.go:28] interesting pod/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 13 17:44:31.504226 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:31.504075 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" podUID="81f574a6-06f5-406b-b8a3-436d157dd1e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 13 17:44:32.504796 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:32.504754 2564 patch_prober.go:28] interesting pod/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 13 17:44:32.505414 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:32.505381 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" podUID="81f574a6-06f5-406b-b8a3-436d157dd1e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 13 17:44:33.504808 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:33.504763 2564 patch_prober.go:28] interesting pod/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 13 17:44:33.505223 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:33.504829 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" podUID="81f574a6-06f5-406b-b8a3-436d157dd1e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 13 17:44:34.508598 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:34.508558 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:34.508996 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:34.508944 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:34.509501 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:34.509483 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:44:37.912169 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:37.912133 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:38.014977 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:38.014938 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:38.735304 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:38.735269 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:43.758194 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:43.758160 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:44:43.900247 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:43.900217 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:49.920452 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:49.920416 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:50.018222 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:50.018189 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:54.370556 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:54.370517 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h"] Jul 13 17:44:54.371078 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:54.370777 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" podUID="dd45853b-4390-44a7-bae7-1549f7d75dd1" containerName="istio-proxy" containerID="cri-o://bd3df9a44555dff4f32e520e09bb44148a115b1020a8ab3998df39f625303d99" gracePeriod=30 Jul 13 17:44:54.392644 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:54.392616 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:54.624891 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:54.624794 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:44:57.996543 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:57.996502 2564 patch_prober.go:28] interesting pod/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 13 17:44:57.996965 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:57.996572 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" podUID="dd45853b-4390-44a7-bae7-1549f7d75dd1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 17:44:59.625835 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.625810 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:44:59.731250 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.731208 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd45853b-4390-44a7-bae7-1549f7d75dd1-istiod-ca-cert\") pod \"dd45853b-4390-44a7-bae7-1549f7d75dd1\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " Jul 13 17:44:59.731431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.731261 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-token\") pod \"dd45853b-4390-44a7-bae7-1549f7d75dd1\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " Jul 13 17:44:59.731431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.731306 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-workload-socket\") pod \"dd45853b-4390-44a7-bae7-1549f7d75dd1\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " Jul 13 17:44:59.731431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.731403 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-data\") pod \"dd45853b-4390-44a7-bae7-1549f7d75dd1\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " Jul 13 17:44:59.731608 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.731491 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-podinfo\") pod \"dd45853b-4390-44a7-bae7-1549f7d75dd1\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " Jul 13 17:44:59.731608 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.731523 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-workload-certs\") pod \"dd45853b-4390-44a7-bae7-1549f7d75dd1\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " Jul 13 17:44:59.731608 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.731571 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-envoy\") pod \"dd45853b-4390-44a7-bae7-1549f7d75dd1\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " Jul 13 17:44:59.731805 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.731607 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tms4\" (UniqueName: \"kubernetes.io/projected/dd45853b-4390-44a7-bae7-1549f7d75dd1-kube-api-access-4tms4\") pod \"dd45853b-4390-44a7-bae7-1549f7d75dd1\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " Jul 13 17:44:59.731805 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.731604 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dd45853b-4390-44a7-bae7-1549f7d75dd1" (UID: "dd45853b-4390-44a7-bae7-1549f7d75dd1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:44:59.731805 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.731631 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-credential-socket\") pod \"dd45853b-4390-44a7-bae7-1549f7d75dd1\" (UID: \"dd45853b-4390-44a7-bae7-1549f7d75dd1\") " Jul 13 17:44:59.731805 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.731768 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd45853b-4390-44a7-bae7-1549f7d75dd1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dd45853b-4390-44a7-bae7-1549f7d75dd1" (UID: "dd45853b-4390-44a7-bae7-1549f7d75dd1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:44:59.732036 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.731891 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dd45853b-4390-44a7-bae7-1549f7d75dd1" (UID: "dd45853b-4390-44a7-bae7-1549f7d75dd1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:44:59.732036 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.731996 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dd45853b-4390-44a7-bae7-1549f7d75dd1" (UID: "dd45853b-4390-44a7-bae7-1549f7d75dd1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:44:59.732143 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.732043 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-workload-certs\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:44:59.732143 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.732062 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd45853b-4390-44a7-bae7-1549f7d75dd1-istiod-ca-cert\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:44:59.732143 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.732076 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-workload-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:44:59.732532 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.732416 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-data" (OuterVolumeSpecName: "istio-data") pod "dd45853b-4390-44a7-bae7-1549f7d75dd1" (UID: "dd45853b-4390-44a7-bae7-1549f7d75dd1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:44:59.733823 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.733790 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dd45853b-4390-44a7-bae7-1549f7d75dd1" (UID: "dd45853b-4390-44a7-bae7-1549f7d75dd1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:44:59.733823 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.733796 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-token" (OuterVolumeSpecName: "istio-token") pod "dd45853b-4390-44a7-bae7-1549f7d75dd1" (UID: "dd45853b-4390-44a7-bae7-1549f7d75dd1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:44:59.733962 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.733816 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dd45853b-4390-44a7-bae7-1549f7d75dd1" (UID: "dd45853b-4390-44a7-bae7-1549f7d75dd1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:44:59.733962 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.733928 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd45853b-4390-44a7-bae7-1549f7d75dd1-kube-api-access-4tms4" (OuterVolumeSpecName: "kube-api-access-4tms4") pod "dd45853b-4390-44a7-bae7-1549f7d75dd1" (UID: "dd45853b-4390-44a7-bae7-1549f7d75dd1"). InnerVolumeSpecName "kube-api-access-4tms4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:44:59.832920 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.832883 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-token\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:44:59.832920 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.832914 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-data\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:44:59.832920 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.832922 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-podinfo\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:44:59.832920 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.832931 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-istio-envoy\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:44:59.833208 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.832941 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tms4\" (UniqueName: \"kubernetes.io/projected/dd45853b-4390-44a7-bae7-1549f7d75dd1-kube-api-access-4tms4\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:44:59.833208 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:44:59.832950 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd45853b-4390-44a7-bae7-1549f7d75dd1-credential-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:45:00.281246 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:00.281209 2564 generic.go:356] "Generic (PLEG): container finished" podID="dd45853b-4390-44a7-bae7-1549f7d75dd1" containerID="bd3df9a44555dff4f32e520e09bb44148a115b1020a8ab3998df39f625303d99" exitCode=0 Jul 13 17:45:00.281428 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:00.281260 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" Jul 13 17:45:00.281428 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:00.281278 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" event={"ID":"dd45853b-4390-44a7-bae7-1549f7d75dd1","Type":"ContainerDied","Data":"bd3df9a44555dff4f32e520e09bb44148a115b1020a8ab3998df39f625303d99"} Jul 13 17:45:00.281428 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:00.281324 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h" event={"ID":"dd45853b-4390-44a7-bae7-1549f7d75dd1","Type":"ContainerDied","Data":"35c12eef18a2a4c7b2c0b4caaba936f35166c5ccf8a532fd69ef62c5656b5331"} Jul 13 17:45:00.281428 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:00.281344 2564 scope.go:117] "RemoveContainer" containerID="bd3df9a44555dff4f32e520e09bb44148a115b1020a8ab3998df39f625303d99" Jul 13 17:45:00.292073 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:00.292054 2564 scope.go:117] "RemoveContainer" containerID="a167ba91de22d937352c5d060a5896bb50b5a1ab9a2ebe93631f4b42442bb725" Jul 13 17:45:00.300239 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:00.300218 2564 scope.go:117] "RemoveContainer" containerID="bd3df9a44555dff4f32e520e09bb44148a115b1020a8ab3998df39f625303d99" Jul 13 17:45:00.300508 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:45:00.300488 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd3df9a44555dff4f32e520e09bb44148a115b1020a8ab3998df39f625303d99\": container with ID starting with bd3df9a44555dff4f32e520e09bb44148a115b1020a8ab3998df39f625303d99 not found: ID does not exist" containerID="bd3df9a44555dff4f32e520e09bb44148a115b1020a8ab3998df39f625303d99" Jul 13 17:45:00.300581 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:00.300521 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd3df9a44555dff4f32e520e09bb44148a115b1020a8ab3998df39f625303d99"} err="failed to get container status \"bd3df9a44555dff4f32e520e09bb44148a115b1020a8ab3998df39f625303d99\": rpc error: code = NotFound desc = could not find container \"bd3df9a44555dff4f32e520e09bb44148a115b1020a8ab3998df39f625303d99\": container with ID starting with bd3df9a44555dff4f32e520e09bb44148a115b1020a8ab3998df39f625303d99 not found: ID does not exist" Jul 13 17:45:00.300581 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:00.300547 2564 scope.go:117] "RemoveContainer" containerID="a167ba91de22d937352c5d060a5896bb50b5a1ab9a2ebe93631f4b42442bb725" Jul 13 17:45:00.300880 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:45:00.300862 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a167ba91de22d937352c5d060a5896bb50b5a1ab9a2ebe93631f4b42442bb725\": container with ID starting with a167ba91de22d937352c5d060a5896bb50b5a1ab9a2ebe93631f4b42442bb725 not found: ID does not exist" containerID="a167ba91de22d937352c5d060a5896bb50b5a1ab9a2ebe93631f4b42442bb725" Jul 13 17:45:00.300928 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:00.300886 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a167ba91de22d937352c5d060a5896bb50b5a1ab9a2ebe93631f4b42442bb725"} err="failed to get container status \"a167ba91de22d937352c5d060a5896bb50b5a1ab9a2ebe93631f4b42442bb725\": rpc error: code = NotFound desc = could not find container \"a167ba91de22d937352c5d060a5896bb50b5a1ab9a2ebe93631f4b42442bb725\": container with ID starting with a167ba91de22d937352c5d060a5896bb50b5a1ab9a2ebe93631f4b42442bb725 not found: ID does not exist" Jul 13 17:45:00.314630 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:00.314606 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h"] Jul 13 17:45:00.322295 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:00.322270 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-edd1bf-openshift-default-68f584d779-s6w4h"] Jul 13 17:45:00.961938 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:00.961907 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd45853b-4390-44a7-bae7-1549f7d75dd1" path="/var/lib/kubelet/pods/dd45853b-4390-44a7-bae7-1549f7d75dd1/volumes" Jul 13 17:45:05.347327 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:05.347295 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:09.035367 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:09.035329 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:45:09.882055 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:09.882022 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:16.192566 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:16.192534 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:16.279118 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:16.279080 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:20.783568 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:20.783533 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs"] Jul 13 17:45:20.784073 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:20.783850 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" podUID="81f574a6-06f5-406b-b8a3-436d157dd1e1" containerName="istio-proxy" containerID="cri-o://f8b9bd6eb2b1ef20d3855797f110a3ffd0c71e5eaafd451469455eb33b434848" gracePeriod=30 Jul 13 17:45:20.793965 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:20.793939 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:21.031245 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.031206 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:21.432790 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.432753 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:21.773054 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.772964 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn"] Jul 13 17:45:21.773816 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.773790 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd45853b-4390-44a7-bae7-1549f7d75dd1" containerName="istio-proxy" Jul 13 17:45:21.773816 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.773816 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd45853b-4390-44a7-bae7-1549f7d75dd1" containerName="istio-proxy" Jul 13 17:45:21.773960 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.773852 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd45853b-4390-44a7-bae7-1549f7d75dd1" containerName="istio-proxy" Jul 13 17:45:21.773960 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.773860 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd45853b-4390-44a7-bae7-1549f7d75dd1" containerName="istio-proxy" Jul 13 17:45:21.773960 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.773946 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd45853b-4390-44a7-bae7-1549f7d75dd1" containerName="istio-proxy" Jul 13 17:45:21.773960 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.773957 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd45853b-4390-44a7-bae7-1549f7d75dd1" containerName="istio-proxy" Jul 13 17:45:21.774091 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.773970 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c1e7060-1f92-4bf4-a713-d92eb1dd61e2" containerName="istio-proxy" Jul 13 17:45:21.777480 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.777455 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.781145 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.781123 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-714452-openshift-default-dockercfg-5bczr\"" Jul 13 17:45:21.790194 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.790170 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn"] Jul 13 17:45:21.815369 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.815335 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:21.834125 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.834096 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-data\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.834280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.834133 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-credential-socket\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.834280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.834153 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7sfs\" (UniqueName: \"kubernetes.io/projected/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-kube-api-access-n7sfs\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.834280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.834175 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-podinfo\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.834280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.834201 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-token\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.834280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.834219 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-envoy\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.834280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.834238 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-workload-certs\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.834280 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.834263 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-workload-socket\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.834530 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.834284 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istiod-ca-cert\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.935404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.935360 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-envoy\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.935404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.935407 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-workload-certs\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.935760 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.935531 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-workload-socket\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.935760 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.935583 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istiod-ca-cert\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.935900 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.935815 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-workload-certs\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.935900 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.935833 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-data\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.935900 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.935885 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-credential-socket\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.936066 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.935919 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7sfs\" (UniqueName: \"kubernetes.io/projected/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-kube-api-access-n7sfs\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.936066 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.935944 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-podinfo\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.936066 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.935962 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-workload-socket\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.936066 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.935970 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-token\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.936274 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.936115 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-data\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.936437 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.936399 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-credential-socket\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.936652 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.936622 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istiod-ca-cert\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.938109 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.938086 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-envoy\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.938657 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.938635 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-podinfo\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.949302 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.949280 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7sfs\" (UniqueName: \"kubernetes.io/projected/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-kube-api-access-n7sfs\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.953229 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.953199 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-token\") pod \"e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:21.990543 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:21.990514 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:22.089507 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:22.089415 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:22.245424 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:22.245286 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn"] Jul 13 17:45:22.248120 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:45:22.248089 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59c7ae08_50d3_4dee_ad3d_ccebd837bd35.slice/crio-4ca45c864d0dd29049b08f28878ba55c80eaffce472742cada4e434f2ed9162e WatchSource:0}: Error finding container 4ca45c864d0dd29049b08f28878ba55c80eaffce472742cada4e434f2ed9162e: Status 404 returned error can't find the container with id 4ca45c864d0dd29049b08f28878ba55c80eaffce472742cada4e434f2ed9162e Jul 13 17:45:22.250225 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:22.250187 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:45:22.250318 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:22.250255 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:45:22.250318 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:22.250289 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:45:22.419292 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:22.419246 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" event={"ID":"59c7ae08-50d3-4dee-ad3d-ccebd837bd35","Type":"ContainerStarted","Data":"748719dcf2d51699f918ea0b4c2e7bb238386b6e0503fe6f6757cd9139a34bcc"} Jul 13 17:45:22.419505 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:22.419324 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" event={"ID":"59c7ae08-50d3-4dee-ad3d-ccebd837bd35","Type":"ContainerStarted","Data":"4ca45c864d0dd29049b08f28878ba55c80eaffce472742cada4e434f2ed9162e"} Jul 13 17:45:22.444922 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:22.444874 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" podStartSLOduration=1.444858242 podStartE2EDuration="1.444858242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:45:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:45:22.443376937 +0000 UTC m=+2400.106152125" watchObservedRunningTime="2026-07-13 17:45:22.444858242 +0000 UTC m=+2400.107633430" Jul 13 17:45:23.089635 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:23.089601 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:23.095565 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:23.095538 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:23.425050 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:23.425020 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:23.426181 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:23.426160 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:24.543501 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:24.543452 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:24.645063 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:24.645011 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:24.743754 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:24.743658 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:24.857312 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:24.857233 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:25.370874 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:25.370832 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:25.474128 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:25.474094 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:26.043590 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.043520 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:45:26.075956 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.075926 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-token\") pod \"81f574a6-06f5-406b-b8a3-436d157dd1e1\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " Jul 13 17:45:26.076135 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.075982 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81f574a6-06f5-406b-b8a3-436d157dd1e1-istiod-ca-cert\") pod \"81f574a6-06f5-406b-b8a3-436d157dd1e1\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " Jul 13 17:45:26.076135 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.076019 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrjnw\" (UniqueName: \"kubernetes.io/projected/81f574a6-06f5-406b-b8a3-436d157dd1e1-kube-api-access-qrjnw\") pod \"81f574a6-06f5-406b-b8a3-436d157dd1e1\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " Jul 13 17:45:26.076135 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.076042 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-podinfo\") pod \"81f574a6-06f5-406b-b8a3-436d157dd1e1\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " Jul 13 17:45:26.076135 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.076087 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-credential-socket\") pod \"81f574a6-06f5-406b-b8a3-436d157dd1e1\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " Jul 13 17:45:26.076348 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.076189 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-workload-certs\") pod \"81f574a6-06f5-406b-b8a3-436d157dd1e1\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " Jul 13 17:45:26.076348 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.076215 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-workload-socket\") pod \"81f574a6-06f5-406b-b8a3-436d157dd1e1\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " Jul 13 17:45:26.076348 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.076251 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-data\") pod \"81f574a6-06f5-406b-b8a3-436d157dd1e1\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " Jul 13 17:45:26.076348 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.076283 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-envoy\") pod \"81f574a6-06f5-406b-b8a3-436d157dd1e1\" (UID: \"81f574a6-06f5-406b-b8a3-436d157dd1e1\") " Jul 13 17:45:26.076896 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.076756 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "81f574a6-06f5-406b-b8a3-436d157dd1e1" (UID: "81f574a6-06f5-406b-b8a3-436d157dd1e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:45:26.076896 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.076758 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "81f574a6-06f5-406b-b8a3-436d157dd1e1" (UID: "81f574a6-06f5-406b-b8a3-436d157dd1e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:45:26.076896 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.076841 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "81f574a6-06f5-406b-b8a3-436d157dd1e1" (UID: "81f574a6-06f5-406b-b8a3-436d157dd1e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:45:26.076896 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.076871 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81f574a6-06f5-406b-b8a3-436d157dd1e1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "81f574a6-06f5-406b-b8a3-436d157dd1e1" (UID: "81f574a6-06f5-406b-b8a3-436d157dd1e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:45:26.077843 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.077813 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-data" (OuterVolumeSpecName: "istio-data") pod "81f574a6-06f5-406b-b8a3-436d157dd1e1" (UID: "81f574a6-06f5-406b-b8a3-436d157dd1e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:45:26.078682 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.078649 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81f574a6-06f5-406b-b8a3-436d157dd1e1-kube-api-access-qrjnw" (OuterVolumeSpecName: "kube-api-access-qrjnw") pod "81f574a6-06f5-406b-b8a3-436d157dd1e1" (UID: "81f574a6-06f5-406b-b8a3-436d157dd1e1"). InnerVolumeSpecName "kube-api-access-qrjnw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:45:26.078818 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.078763 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-token" (OuterVolumeSpecName: "istio-token") pod "81f574a6-06f5-406b-b8a3-436d157dd1e1" (UID: "81f574a6-06f5-406b-b8a3-436d157dd1e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:45:26.078891 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.078871 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "81f574a6-06f5-406b-b8a3-436d157dd1e1" (UID: "81f574a6-06f5-406b-b8a3-436d157dd1e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:45:26.078959 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.078907 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "81f574a6-06f5-406b-b8a3-436d157dd1e1" (UID: "81f574a6-06f5-406b-b8a3-436d157dd1e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:45:26.177469 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.177431 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-credential-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:45:26.177469 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.177462 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-workload-certs\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:45:26.177469 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.177471 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-workload-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:45:26.177712 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.177480 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-data\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:45:26.177712 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.177490 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-envoy\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:45:26.177712 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.177497 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-token\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:45:26.177712 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.177504 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81f574a6-06f5-406b-b8a3-436d157dd1e1-istiod-ca-cert\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:45:26.177712 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.177512 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrjnw\" (UniqueName: \"kubernetes.io/projected/81f574a6-06f5-406b-b8a3-436d157dd1e1-kube-api-access-qrjnw\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:45:26.177712 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.177519 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81f574a6-06f5-406b-b8a3-436d157dd1e1-istio-podinfo\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:45:26.442846 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.442806 2564 generic.go:356] "Generic (PLEG): container finished" podID="81f574a6-06f5-406b-b8a3-436d157dd1e1" containerID="f8b9bd6eb2b1ef20d3855797f110a3ffd0c71e5eaafd451469455eb33b434848" exitCode=0 Jul 13 17:45:26.443045 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.442857 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" Jul 13 17:45:26.443045 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.442892 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" event={"ID":"81f574a6-06f5-406b-b8a3-436d157dd1e1","Type":"ContainerDied","Data":"f8b9bd6eb2b1ef20d3855797f110a3ffd0c71e5eaafd451469455eb33b434848"} Jul 13 17:45:26.443045 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.442948 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs" event={"ID":"81f574a6-06f5-406b-b8a3-436d157dd1e1","Type":"ContainerDied","Data":"463ca466962d14e27a234eebc65e6047e152fa02db2722d8dde168ca9c3ec231"} Jul 13 17:45:26.443045 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.442981 2564 scope.go:117] "RemoveContainer" containerID="f8b9bd6eb2b1ef20d3855797f110a3ffd0c71e5eaafd451469455eb33b434848" Jul 13 17:45:26.452887 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.452865 2564 scope.go:117] "RemoveContainer" containerID="f53a68f8665d32819ebb9ea605cf4ac28b9ccb48f2820769f42a5168aa579714" Jul 13 17:45:26.461701 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.461681 2564 scope.go:117] "RemoveContainer" containerID="f8b9bd6eb2b1ef20d3855797f110a3ffd0c71e5eaafd451469455eb33b434848" Jul 13 17:45:26.461991 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:45:26.461970 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8b9bd6eb2b1ef20d3855797f110a3ffd0c71e5eaafd451469455eb33b434848\": container with ID starting with f8b9bd6eb2b1ef20d3855797f110a3ffd0c71e5eaafd451469455eb33b434848 not found: ID does not exist" containerID="f8b9bd6eb2b1ef20d3855797f110a3ffd0c71e5eaafd451469455eb33b434848" Jul 13 17:45:26.462030 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.462002 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8b9bd6eb2b1ef20d3855797f110a3ffd0c71e5eaafd451469455eb33b434848"} err="failed to get container status \"f8b9bd6eb2b1ef20d3855797f110a3ffd0c71e5eaafd451469455eb33b434848\": rpc error: code = NotFound desc = could not find container \"f8b9bd6eb2b1ef20d3855797f110a3ffd0c71e5eaafd451469455eb33b434848\": container with ID starting with f8b9bd6eb2b1ef20d3855797f110a3ffd0c71e5eaafd451469455eb33b434848 not found: ID does not exist" Jul 13 17:45:26.462030 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.462022 2564 scope.go:117] "RemoveContainer" containerID="f53a68f8665d32819ebb9ea605cf4ac28b9ccb48f2820769f42a5168aa579714" Jul 13 17:45:26.462254 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:45:26.462235 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f53a68f8665d32819ebb9ea605cf4ac28b9ccb48f2820769f42a5168aa579714\": container with ID starting with f53a68f8665d32819ebb9ea605cf4ac28b9ccb48f2820769f42a5168aa579714 not found: ID does not exist" containerID="f53a68f8665d32819ebb9ea605cf4ac28b9ccb48f2820769f42a5168aa579714" Jul 13 17:45:26.462366 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.462257 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f53a68f8665d32819ebb9ea605cf4ac28b9ccb48f2820769f42a5168aa579714"} err="failed to get container status \"f53a68f8665d32819ebb9ea605cf4ac28b9ccb48f2820769f42a5168aa579714\": rpc error: code = NotFound desc = could not find container \"f53a68f8665d32819ebb9ea605cf4ac28b9ccb48f2820769f42a5168aa579714\": container with ID starting with f53a68f8665d32819ebb9ea605cf4ac28b9ccb48f2820769f42a5168aa579714 not found: ID does not exist" Jul 13 17:45:26.469084 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.469052 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs"] Jul 13 17:45:26.471387 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.471366 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-e386c8-openshift-default-6d9c756747-lp2rs"] Jul 13 17:45:26.960990 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:26.960956 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81f574a6-06f5-406b-b8a3-436d157dd1e1" path="/var/lib/kubelet/pods/81f574a6-06f5-406b-b8a3-436d157dd1e1/volumes" Jul 13 17:45:29.135898 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.135857 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:29.513337 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.513301 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:29.644913 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.644877 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:45:29.820193 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.820108 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h"] Jul 13 17:45:29.820543 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.820525 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81f574a6-06f5-406b-b8a3-436d157dd1e1" containerName="istio-proxy" Jul 13 17:45:29.820622 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.820545 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="81f574a6-06f5-406b-b8a3-436d157dd1e1" containerName="istio-proxy" Jul 13 17:45:29.820622 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.820568 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81f574a6-06f5-406b-b8a3-436d157dd1e1" containerName="istio-proxy" Jul 13 17:45:29.820622 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.820577 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="81f574a6-06f5-406b-b8a3-436d157dd1e1" containerName="istio-proxy" Jul 13 17:45:29.820812 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.820692 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="81f574a6-06f5-406b-b8a3-436d157dd1e1" containerName="istio-proxy" Jul 13 17:45:29.820812 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.820705 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="81f574a6-06f5-406b-b8a3-436d157dd1e1" containerName="istio-proxy" Jul 13 17:45:29.823363 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.823338 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:29.830463 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.830442 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-31a784-openshift-default-dockercfg-c6mmx\"" Jul 13 17:45:29.839164 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.839140 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h"] Jul 13 17:45:29.858507 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.858481 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:29.908933 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.908899 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-data\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:29.908933 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.908946 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqhrl\" (UniqueName: \"kubernetes.io/projected/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-kube-api-access-jqhrl\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:29.909203 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.908973 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-workload-socket\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:29.909203 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.909084 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-envoy\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:29.909203 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.909128 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-credential-socket\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:29.909203 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.909171 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istiod-ca-cert\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:29.909366 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.909239 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-token\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:29.909366 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.909287 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-workload-certs\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:29.909366 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:29.909308 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-podinfo\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.010016 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.009984 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-data\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.010259 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.010029 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqhrl\" (UniqueName: \"kubernetes.io/projected/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-kube-api-access-jqhrl\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.010259 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.010085 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-workload-socket\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.010259 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.010122 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-envoy\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.010259 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.010160 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-credential-socket\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.010259 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.010190 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istiod-ca-cert\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.010259 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.010217 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-token\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.010604 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.010264 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-workload-certs\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.010803 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.010772 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-podinfo\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.010965 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.010944 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-data\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.011044 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.010964 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istiod-ca-cert\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.011419 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.011392 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-credential-socket\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.011577 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.011553 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-workload-socket\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.011774 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.011450 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-workload-certs\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.013444 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.013382 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-envoy\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.014029 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.013990 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-podinfo\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.019828 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.019797 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-token\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.021054 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.021026 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqhrl\" (UniqueName: \"kubernetes.io/projected/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-kube-api-access-jqhrl\") pod \"e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.133204 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.133111 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:30.165027 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.164993 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:30.296157 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.296125 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h"] Jul 13 17:45:30.298029 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:45:30.298003 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ed40df3_56ea_4da2_a02e_05a5aa2b84ff.slice/crio-2fd28ed3d4c4176825e6dbb8fe7f0237710dd7b814efd8042f270e8d5f77f8bf WatchSource:0}: Error finding container 2fd28ed3d4c4176825e6dbb8fe7f0237710dd7b814efd8042f270e8d5f77f8bf: Status 404 returned error can't find the container with id 2fd28ed3d4c4176825e6dbb8fe7f0237710dd7b814efd8042f270e8d5f77f8bf Jul 13 17:45:30.300559 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.300529 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:45:30.300621 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.300597 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:45:30.300656 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.300627 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:45:30.473746 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.473697 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" event={"ID":"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff","Type":"ContainerStarted","Data":"b585300960bd70e9ad595f703879ab0b05835559b5c1be7ab54ddeeb54e1cc52"} Jul 13 17:45:30.473939 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.473821 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" event={"ID":"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff","Type":"ContainerStarted","Data":"2fd28ed3d4c4176825e6dbb8fe7f0237710dd7b814efd8042f270e8d5f77f8bf"} Jul 13 17:45:30.503691 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:30.503636 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" podStartSLOduration=1.503617921 podStartE2EDuration="1.503617921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:45:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:45:30.500382643 +0000 UTC m=+2408.163157832" watchObservedRunningTime="2026-07-13 17:45:30.503617921 +0000 UTC m=+2408.166393108" Jul 13 17:45:31.133614 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:31.133575 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:31.138288 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:31.138258 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:31.483935 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:31.483897 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:31.489273 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:31.489236 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:32.510516 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:32.510466 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:32.619114 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:32.619060 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:32.744998 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:32.744959 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:32.919939 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:32.919903 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:33.463561 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:33.463511 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:33.565365 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:33.565330 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:37.101480 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:37.101449 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:38.426450 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:38.426411 2564 patch_prober.go:28] interesting pod/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 13 17:45:38.426989 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:38.426488 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" podUID="59c7ae08-50d3-4dee-ad3d-ccebd837bd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 13 17:45:38.533176 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:38.533138 2564 generic.go:356] "Generic (PLEG): container finished" podID="59c7ae08-50d3-4dee-ad3d-ccebd837bd35" containerID="748719dcf2d51699f918ea0b4c2e7bb238386b6e0503fe6f6757cd9139a34bcc" exitCode=0 Jul 13 17:45:38.533176 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:38.533178 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" event={"ID":"59c7ae08-50d3-4dee-ad3d-ccebd837bd35","Type":"ContainerDied","Data":"748719dcf2d51699f918ea0b4c2e7bb238386b6e0503fe6f6757cd9139a34bcc"} Jul 13 17:45:38.533588 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:38.533575 2564 scope.go:117] "RemoveContainer" containerID="748719dcf2d51699f918ea0b4c2e7bb238386b6e0503fe6f6757cd9139a34bcc" Jul 13 17:45:38.534601 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:38.534553 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:45:38.534679 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:38.534640 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:45:38.534679 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:38.534669 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:45:39.540541 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:39.540502 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" event={"ID":"59c7ae08-50d3-4dee-ad3d-ccebd837bd35","Type":"ContainerStarted","Data":"eb202907e196b4007bd0436de1d92601ef315280004cbdb2e3a3ba88c57ba2d4"} Jul 13 17:45:40.093756 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:40.092204 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:40.093756 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:40.092347 2564 patch_prober.go:28] interesting pod/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 13 17:45:40.093756 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:40.092393 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" podUID="59c7ae08-50d3-4dee-ad3d-ccebd837bd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 13 17:45:41.090443 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:41.090403 2564 patch_prober.go:28] interesting pod/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 13 17:45:41.091529 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:41.091079 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" podUID="59c7ae08-50d3-4dee-ad3d-ccebd837bd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 13 17:45:42.091125 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:42.090524 2564 patch_prober.go:28] interesting pod/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 13 17:45:42.091125 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:42.090606 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" podUID="59c7ae08-50d3-4dee-ad3d-ccebd837bd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 13 17:45:43.090802 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:43.090344 2564 patch_prober.go:28] interesting pod/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 13 17:45:43.090802 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:43.090407 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" podUID="59c7ae08-50d3-4dee-ad3d-ccebd837bd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 13 17:45:44.094877 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:44.094838 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:44.095408 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:44.095384 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:44.096168 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:44.096148 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:45:45.578911 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:45.578825 2564 generic.go:356] "Generic (PLEG): container finished" podID="0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" containerID="b585300960bd70e9ad595f703879ab0b05835559b5c1be7ab54ddeeb54e1cc52" exitCode=0 Jul 13 17:45:45.578911 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:45.578902 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" event={"ID":"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff","Type":"ContainerDied","Data":"b585300960bd70e9ad595f703879ab0b05835559b5c1be7ab54ddeeb54e1cc52"} Jul 13 17:45:45.579514 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:45.579494 2564 scope.go:117] "RemoveContainer" containerID="b585300960bd70e9ad595f703879ab0b05835559b5c1be7ab54ddeeb54e1cc52" Jul 13 17:45:45.580569 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:45.580533 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:45:45.580665 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:45.580614 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:45:45.580665 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:45.580659 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 17:45:46.037083 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:46.037036 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:46.139097 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:46.139046 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:46.248762 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:46.248715 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:45:46.484087 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:46.484046 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:46.591484 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:46.591446 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" event={"ID":"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff","Type":"ContainerStarted","Data":"5df2bed5ab966acfc0548bc995c80ef91307d4c2cb0572810255c6dd94e28dac"} Jul 13 17:45:46.924051 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:46.924015 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:47.031650 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:47.031617 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:47.133523 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:47.133478 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:47.134870 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:47.134838 2564 patch_prober.go:28] interesting pod/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 13 17:45:47.135024 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:47.134891 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" podUID="0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 13 17:45:48.134378 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:48.134343 2564 patch_prober.go:28] interesting pod/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 13 17:45:48.134787 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:48.134399 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" podUID="0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 13 17:45:49.134052 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:49.134012 2564 patch_prober.go:28] interesting pod/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 13 17:45:49.134253 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:49.134075 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" podUID="0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 13 17:45:50.133570 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:50.133534 2564 patch_prober.go:28] interesting pod/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 13 17:45:50.134062 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:50.133587 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" podUID="0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 13 17:45:51.138108 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:51.138073 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:51.138515 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:51.138391 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:51.139024 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:51.139005 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:45:51.370836 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:51.370788 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:54.380801 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:54.380761 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:54.661370 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:54.661333 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:45:55.745945 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:45:55.745901 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:46:33.238550 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:46:33.238511 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:46:36.749948 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:46:36.749910 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:46:53.911409 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:46:53.911368 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:46:54.730081 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:46:54.730040 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:47:05.602690 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:47:05.602650 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:47:13.741067 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:47:13.740959 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:47:22.505151 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:47:22.505114 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:47:22.576929 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:47:22.576890 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:47:24.924891 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:47:24.924850 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:47:31.784862 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:47:31.784822 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:47:35.890474 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:47:35.890416 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:47:46.555666 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:47:46.555623 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:48:03.757928 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:48:03.757888 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:48:19.141157 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:48:19.141116 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:48:38.936214 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:48:38.936178 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:48:42.005424 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:48:42.005388 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:48:42.998267 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:48:42.998228 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:48:54.090498 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:48:54.090456 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:48:54.841316 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:48:54.841273 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:49:06.643162 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:49:06.643122 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:49:18.087868 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:49:18.087824 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:49:22.680184 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:49:22.680147 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:49:52.280824 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:49:52.280698 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:50:01.457663 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:50:01.457625 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:50:16.719724 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:50:16.719598 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:50:17.573200 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:50:17.573163 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:50:20.929122 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:50:20.929074 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:50:22.392232 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:50:22.392191 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:50:38.035498 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:50:38.035458 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:50:46.999644 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:50:46.999608 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:50:47.100327 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:50:47.100291 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:50:47.862331 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:50:47.862301 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:50:47.962342 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:50:47.962312 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:50:55.384085 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:50:55.384043 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:50:55.484871 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:50:55.484838 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:50:56.252085 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:50:56.252048 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:51:00.358620 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:00.358584 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:51:15.534028 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:15.533991 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:51:31.963711 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:31.963670 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:51:39.047240 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:39.047205 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:51:39.984691 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:39.984646 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:51:40.991091 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:40.991049 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-8292-139830916340912"] Jul 13 17:51:40.993911 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:40.993887 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830916340912" Jul 13 17:51:40.997004 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:40.996985 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-dsztx\"" Jul 13 17:51:41.003401 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:41.003374 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-8292-139830916340912"] Jul 13 17:51:41.031381 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:41.031347 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24hbl\" (UniqueName: \"kubernetes.io/projected/ba686bcb-9a55-450f-b5de-b69c1f5b035f-kube-api-access-24hbl\") pod \"test-curl-8292-139830916340912\" (UID: \"ba686bcb-9a55-450f-b5de-b69c1f5b035f\") " pod="openshift-ingress/test-curl-8292-139830916340912" Jul 13 17:51:41.132096 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:41.132052 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24hbl\" (UniqueName: \"kubernetes.io/projected/ba686bcb-9a55-450f-b5de-b69c1f5b035f-kube-api-access-24hbl\") pod \"test-curl-8292-139830916340912\" (UID: \"ba686bcb-9a55-450f-b5de-b69c1f5b035f\") " pod="openshift-ingress/test-curl-8292-139830916340912" Jul 13 17:51:41.145422 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:41.145383 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24hbl\" (UniqueName: \"kubernetes.io/projected/ba686bcb-9a55-450f-b5de-b69c1f5b035f-kube-api-access-24hbl\") pod \"test-curl-8292-139830916340912\" (UID: \"ba686bcb-9a55-450f-b5de-b69c1f5b035f\") " pod="openshift-ingress/test-curl-8292-139830916340912" Jul 13 17:51:41.302922 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:41.302824 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830916340912" Jul 13 17:51:41.464391 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:41.464366 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-8292-139830916340912"] Jul 13 17:51:41.466606 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:51:41.466561 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba686bcb_9a55_450f_b5de_b69c1f5b035f.slice/crio-ffbc0875767059d7ed1c31da5bf434e961bba8c1811c4dacb2925a9d596ed834 WatchSource:0}: Error finding container ffbc0875767059d7ed1c31da5bf434e961bba8c1811c4dacb2925a9d596ed834: Status 404 returned error can't find the container with id ffbc0875767059d7ed1c31da5bf434e961bba8c1811c4dacb2925a9d596ed834 Jul 13 17:51:41.468398 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:41.468375 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 17:51:41.602487 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:41.602400 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-8292-139830916340912" event={"ID":"ba686bcb-9a55-450f-b5de-b69c1f5b035f","Type":"ContainerStarted","Data":"ffbc0875767059d7ed1c31da5bf434e961bba8c1811c4dacb2925a9d596ed834"} Jul 13 17:51:42.609923 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:42.609826 2564 generic.go:356] "Generic (PLEG): container finished" podID="ba686bcb-9a55-450f-b5de-b69c1f5b035f" containerID="622504b2d67edf89e42cc652af5ca6cf381aaa98d8ef89dda27240e1ff02351b" exitCode=0 Jul 13 17:51:42.610358 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:42.609913 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-8292-139830916340912" event={"ID":"ba686bcb-9a55-450f-b5de-b69c1f5b035f","Type":"ContainerDied","Data":"622504b2d67edf89e42cc652af5ca6cf381aaa98d8ef89dda27240e1ff02351b"} Jul 13 17:51:43.760976 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:43.760943 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830916340912" Jul 13 17:51:43.783570 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:43.783546 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-8292-139830916340912_ba686bcb-9a55-450f-b5de-b69c1f5b035f/test-curl-8292-139830916340912/0.log" Jul 13 17:51:43.811853 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:43.811815 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-8292-139830916340912"] Jul 13 17:51:43.817952 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:43.817925 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-8292-139830916340912"] Jul 13 17:51:43.860849 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:43.860821 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24hbl\" (UniqueName: \"kubernetes.io/projected/ba686bcb-9a55-450f-b5de-b69c1f5b035f-kube-api-access-24hbl\") pod \"ba686bcb-9a55-450f-b5de-b69c1f5b035f\" (UID: \"ba686bcb-9a55-450f-b5de-b69c1f5b035f\") " Jul 13 17:51:43.863165 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:43.863140 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba686bcb-9a55-450f-b5de-b69c1f5b035f-kube-api-access-24hbl" (OuterVolumeSpecName: "kube-api-access-24hbl") pod "ba686bcb-9a55-450f-b5de-b69c1f5b035f" (UID: "ba686bcb-9a55-450f-b5de-b69c1f5b035f"). InnerVolumeSpecName "kube-api-access-24hbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:51:43.952441 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:43.952403 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-8292-139830916134832"] Jul 13 17:51:43.952837 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:43.952821 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba686bcb-9a55-450f-b5de-b69c1f5b035f" containerName="test-curl-8292-139830916340912" Jul 13 17:51:43.952890 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:43.952840 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba686bcb-9a55-450f-b5de-b69c1f5b035f" containerName="test-curl-8292-139830916340912" Jul 13 17:51:43.952925 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:43.952910 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba686bcb-9a55-450f-b5de-b69c1f5b035f" containerName="test-curl-8292-139830916340912" Jul 13 17:51:43.955606 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:43.955582 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830916134832" Jul 13 17:51:43.961803 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:43.961776 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24hbl\" (UniqueName: \"kubernetes.io/projected/ba686bcb-9a55-450f-b5de-b69c1f5b035f-kube-api-access-24hbl\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:51:43.964839 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:43.964813 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-8292-139830916134832"] Jul 13 17:51:44.063309 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:44.063268 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdkz7\" (UniqueName: \"kubernetes.io/projected/bd90f607-ce6e-4b13-a74a-dbacf5912c34-kube-api-access-pdkz7\") pod \"test-curl-8292-139830916134832\" (UID: \"bd90f607-ce6e-4b13-a74a-dbacf5912c34\") " pod="openshift-ingress/test-curl-8292-139830916134832" Jul 13 17:51:44.164181 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:44.164143 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdkz7\" (UniqueName: \"kubernetes.io/projected/bd90f607-ce6e-4b13-a74a-dbacf5912c34-kube-api-access-pdkz7\") pod \"test-curl-8292-139830916134832\" (UID: \"bd90f607-ce6e-4b13-a74a-dbacf5912c34\") " pod="openshift-ingress/test-curl-8292-139830916134832" Jul 13 17:51:44.174049 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:44.174023 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdkz7\" (UniqueName: \"kubernetes.io/projected/bd90f607-ce6e-4b13-a74a-dbacf5912c34-kube-api-access-pdkz7\") pod \"test-curl-8292-139830916134832\" (UID: \"bd90f607-ce6e-4b13-a74a-dbacf5912c34\") " pod="openshift-ingress/test-curl-8292-139830916134832" Jul 13 17:51:44.265368 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:44.265320 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830916134832" Jul 13 17:51:44.410971 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:44.407777 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-8292-139830916134832"] Jul 13 17:51:44.623102 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:44.623063 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830916340912" Jul 13 17:51:44.623102 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:44.623084 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ffbc0875767059d7ed1c31da5bf434e961bba8c1811c4dacb2925a9d596ed834" Jul 13 17:51:44.624361 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:44.624328 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-8292-139830916134832" event={"ID":"bd90f607-ce6e-4b13-a74a-dbacf5912c34","Type":"ContainerStarted","Data":"09224ad9d38f8ce069b64b2615e7b90bbad39d712a5efed7fcaa1b80daec603b"} Jul 13 17:51:44.634294 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:51:44.634268 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8292-139830916340912\" is forbidden: User \"system:node:ip-10-0-137-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-137-245.ec2.internal' and this object" podUID="ba686bcb-9a55-450f-b5de-b69c1f5b035f" pod="openshift-ingress/test-curl-8292-139830916340912" Jul 13 17:51:44.961636 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:44.961601 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba686bcb-9a55-450f-b5de-b69c1f5b035f" path="/var/lib/kubelet/pods/ba686bcb-9a55-450f-b5de-b69c1f5b035f/volumes" Jul 13 17:51:45.631746 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:45.631699 2564 generic.go:356] "Generic (PLEG): container finished" podID="bd90f607-ce6e-4b13-a74a-dbacf5912c34" containerID="4fc6def65c571672bd9386b5253412ecf3813416299af5b1fa17e92f9d55ede3" exitCode=0 Jul 13 17:51:45.631933 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:45.631781 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-8292-139830916134832" event={"ID":"bd90f607-ce6e-4b13-a74a-dbacf5912c34","Type":"ContainerDied","Data":"4fc6def65c571672bd9386b5253412ecf3813416299af5b1fa17e92f9d55ede3"} Jul 13 17:51:46.732579 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:46.732543 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:51:46.789612 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:46.789587 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830916134832" Jul 13 17:51:46.823116 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:46.823081 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-8292-139830916134832_bd90f607-ce6e-4b13-a74a-dbacf5912c34/test-curl-8292-139830916134832/0.log" Jul 13 17:51:46.848016 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:46.847968 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-8292-139830916134832"] Jul 13 17:51:46.855927 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:46.855901 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-8292-139830916134832"] Jul 13 17:51:46.890603 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:46.890519 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdkz7\" (UniqueName: \"kubernetes.io/projected/bd90f607-ce6e-4b13-a74a-dbacf5912c34-kube-api-access-pdkz7\") pod \"bd90f607-ce6e-4b13-a74a-dbacf5912c34\" (UID: \"bd90f607-ce6e-4b13-a74a-dbacf5912c34\") " Jul 13 17:51:46.892914 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:46.892880 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd90f607-ce6e-4b13-a74a-dbacf5912c34-kube-api-access-pdkz7" (OuterVolumeSpecName: "kube-api-access-pdkz7") pod "bd90f607-ce6e-4b13-a74a-dbacf5912c34" (UID: "bd90f607-ce6e-4b13-a74a-dbacf5912c34"). InnerVolumeSpecName "kube-api-access-pdkz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:51:46.961899 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:46.961862 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd90f607-ce6e-4b13-a74a-dbacf5912c34" path="/var/lib/kubelet/pods/bd90f607-ce6e-4b13-a74a-dbacf5912c34/volumes" Jul 13 17:51:46.991819 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:46.991788 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdkz7\" (UniqueName: \"kubernetes.io/projected/bd90f607-ce6e-4b13-a74a-dbacf5912c34-kube-api-access-pdkz7\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:51:47.023185 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:47.023149 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-8292-139830918241136"] Jul 13 17:51:47.023552 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:47.023538 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd90f607-ce6e-4b13-a74a-dbacf5912c34" containerName="test-curl-8292-139830916134832" Jul 13 17:51:47.023552 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:47.023553 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd90f607-ce6e-4b13-a74a-dbacf5912c34" containerName="test-curl-8292-139830916134832" Jul 13 17:51:47.023640 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:47.023621 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd90f607-ce6e-4b13-a74a-dbacf5912c34" containerName="test-curl-8292-139830916134832" Jul 13 17:51:47.026082 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:47.026063 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830918241136" Jul 13 17:51:47.042431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:47.042401 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-8292-139830918241136"] Jul 13 17:51:47.093197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:47.093165 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5cj5\" (UniqueName: \"kubernetes.io/projected/f5caeff2-fdc1-4128-a173-1f7e02b669b2-kube-api-access-n5cj5\") pod \"test-curl-8292-139830918241136\" (UID: \"f5caeff2-fdc1-4128-a173-1f7e02b669b2\") " pod="openshift-ingress/test-curl-8292-139830918241136" Jul 13 17:51:47.194652 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:47.194610 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5cj5\" (UniqueName: \"kubernetes.io/projected/f5caeff2-fdc1-4128-a173-1f7e02b669b2-kube-api-access-n5cj5\") pod \"test-curl-8292-139830918241136\" (UID: \"f5caeff2-fdc1-4128-a173-1f7e02b669b2\") " pod="openshift-ingress/test-curl-8292-139830918241136" Jul 13 17:51:47.206253 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:47.206224 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5cj5\" (UniqueName: \"kubernetes.io/projected/f5caeff2-fdc1-4128-a173-1f7e02b669b2-kube-api-access-n5cj5\") pod \"test-curl-8292-139830918241136\" (UID: \"f5caeff2-fdc1-4128-a173-1f7e02b669b2\") " pod="openshift-ingress/test-curl-8292-139830918241136" Jul 13 17:51:47.335671 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:47.335625 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830918241136" Jul 13 17:51:47.475506 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:47.475479 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-8292-139830918241136"] Jul 13 17:51:47.476615 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:51:47.476592 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5caeff2_fdc1_4128_a173_1f7e02b669b2.slice/crio-97864401e2056428addf1466bd03e647b9642e45f1e6f034e0187a412fa022da WatchSource:0}: Error finding container 97864401e2056428addf1466bd03e647b9642e45f1e6f034e0187a412fa022da: Status 404 returned error can't find the container with id 97864401e2056428addf1466bd03e647b9642e45f1e6f034e0187a412fa022da Jul 13 17:51:47.644806 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:47.644779 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830916134832" Jul 13 17:51:47.645020 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:47.644815 2564 scope.go:117] "RemoveContainer" containerID="4fc6def65c571672bd9386b5253412ecf3813416299af5b1fa17e92f9d55ede3" Jul 13 17:51:47.646996 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:47.646963 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-8292-139830918241136" event={"ID":"f5caeff2-fdc1-4128-a173-1f7e02b669b2","Type":"ContainerStarted","Data":"97864401e2056428addf1466bd03e647b9642e45f1e6f034e0187a412fa022da"} Jul 13 17:51:48.656979 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:48.656927 2564 generic.go:356] "Generic (PLEG): container finished" podID="f5caeff2-fdc1-4128-a173-1f7e02b669b2" containerID="1e1d4f57380a6f187bbbd639bda9e657b2c277dd88a4235a5dc6e771c7737752" exitCode=0 Jul 13 17:51:48.657494 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:48.657111 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-8292-139830918241136" event={"ID":"f5caeff2-fdc1-4128-a173-1f7e02b669b2","Type":"ContainerDied","Data":"1e1d4f57380a6f187bbbd639bda9e657b2c277dd88a4235a5dc6e771c7737752"} Jul 13 17:51:49.810458 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:49.810429 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830918241136" Jul 13 17:51:49.833520 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:49.833496 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-8292-139830918241136_f5caeff2-fdc1-4128-a173-1f7e02b669b2/test-curl-8292-139830918241136/0.log" Jul 13 17:51:49.863577 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:49.863531 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-8292-139830918241136"] Jul 13 17:51:49.873644 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:49.873609 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-8292-139830918241136"] Jul 13 17:51:49.923565 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:49.923462 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5cj5\" (UniqueName: \"kubernetes.io/projected/f5caeff2-fdc1-4128-a173-1f7e02b669b2-kube-api-access-n5cj5\") pod \"f5caeff2-fdc1-4128-a173-1f7e02b669b2\" (UID: \"f5caeff2-fdc1-4128-a173-1f7e02b669b2\") " Jul 13 17:51:49.925743 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:49.925695 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5caeff2-fdc1-4128-a173-1f7e02b669b2-kube-api-access-n5cj5" (OuterVolumeSpecName: "kube-api-access-n5cj5") pod "f5caeff2-fdc1-4128-a173-1f7e02b669b2" (UID: "f5caeff2-fdc1-4128-a173-1f7e02b669b2"). InnerVolumeSpecName "kube-api-access-n5cj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:51:50.001147 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:50.001109 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-8292-139830918241136"] Jul 13 17:51:50.001520 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:50.001505 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5caeff2-fdc1-4128-a173-1f7e02b669b2" containerName="test-curl-8292-139830918241136" Jul 13 17:51:50.001520 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:50.001519 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5caeff2-fdc1-4128-a173-1f7e02b669b2" containerName="test-curl-8292-139830918241136" Jul 13 17:51:50.001686 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:50.001576 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5caeff2-fdc1-4128-a173-1f7e02b669b2" containerName="test-curl-8292-139830918241136" Jul 13 17:51:50.003661 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:50.003642 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830918241136" Jul 13 17:51:50.011529 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:50.011506 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-8292-139830918241136"] Jul 13 17:51:50.024929 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:50.024891 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5cj5\" (UniqueName: \"kubernetes.io/projected/f5caeff2-fdc1-4128-a173-1f7e02b669b2-kube-api-access-n5cj5\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:51:50.125398 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:50.125354 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6hc8\" (UniqueName: \"kubernetes.io/projected/44ea9c34-71b0-4e65-9730-9575d33bf414-kube-api-access-w6hc8\") pod \"test-curl-8292-139830918241136\" (UID: \"44ea9c34-71b0-4e65-9730-9575d33bf414\") " pod="openshift-ingress/test-curl-8292-139830918241136" Jul 13 17:51:50.226018 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:50.225927 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6hc8\" (UniqueName: \"kubernetes.io/projected/44ea9c34-71b0-4e65-9730-9575d33bf414-kube-api-access-w6hc8\") pod \"test-curl-8292-139830918241136\" (UID: \"44ea9c34-71b0-4e65-9730-9575d33bf414\") " pod="openshift-ingress/test-curl-8292-139830918241136" Jul 13 17:51:50.234604 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:50.234575 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6hc8\" (UniqueName: \"kubernetes.io/projected/44ea9c34-71b0-4e65-9730-9575d33bf414-kube-api-access-w6hc8\") pod \"test-curl-8292-139830918241136\" (UID: \"44ea9c34-71b0-4e65-9730-9575d33bf414\") " pod="openshift-ingress/test-curl-8292-139830918241136" Jul 13 17:51:50.313431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:50.313390 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830918241136" Jul 13 17:51:50.450546 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:50.450520 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-8292-139830918241136"] Jul 13 17:51:50.451913 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:51:50.451885 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44ea9c34_71b0_4e65_9730_9575d33bf414.slice/crio-17641be5549c49e6a0920932791ebfeac112efd7b7445d1dfae5f7f2c24d7c38 WatchSource:0}: Error finding container 17641be5549c49e6a0920932791ebfeac112efd7b7445d1dfae5f7f2c24d7c38: Status 404 returned error can't find the container with id 17641be5549c49e6a0920932791ebfeac112efd7b7445d1dfae5f7f2c24d7c38 Jul 13 17:51:50.672607 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:50.672560 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-8292-139830918241136" event={"ID":"44ea9c34-71b0-4e65-9730-9575d33bf414","Type":"ContainerStarted","Data":"17641be5549c49e6a0920932791ebfeac112efd7b7445d1dfae5f7f2c24d7c38"} Jul 13 17:51:50.674110 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:50.674090 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97864401e2056428addf1466bd03e647b9642e45f1e6f034e0187a412fa022da" Jul 13 17:51:50.674233 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:50.674149 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830918241136" Jul 13 17:51:50.686309 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:50.686271 2564 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-8292-139830918241136" oldPodUID="f5caeff2-fdc1-4128-a173-1f7e02b669b2" podUID="44ea9c34-71b0-4e65-9730-9575d33bf414" Jul 13 17:51:50.961557 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:50.961524 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5caeff2-fdc1-4128-a173-1f7e02b669b2" path="/var/lib/kubelet/pods/f5caeff2-fdc1-4128-a173-1f7e02b669b2/volumes" Jul 13 17:51:51.682422 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:51.682383 2564 generic.go:356] "Generic (PLEG): container finished" podID="44ea9c34-71b0-4e65-9730-9575d33bf414" containerID="125856e7be08f946e9f95e2bc22147f1a94357d742c8350518bbf3b387d7bdb0" exitCode=0 Jul 13 17:51:51.682590 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:51.682439 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-8292-139830918241136" event={"ID":"44ea9c34-71b0-4e65-9730-9575d33bf414","Type":"ContainerDied","Data":"125856e7be08f946e9f95e2bc22147f1a94357d742c8350518bbf3b387d7bdb0"} Jul 13 17:51:52.828056 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:52.828031 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830918241136" Jul 13 17:51:52.851492 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:52.851452 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-8292-139830918241136_44ea9c34-71b0-4e65-9730-9575d33bf414/test-curl-8292-139830918241136/0.log" Jul 13 17:51:52.876426 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:52.876389 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-8292-139830918241136"] Jul 13 17:51:52.883566 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:52.883533 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-8292-139830918241136"] Jul 13 17:51:52.950697 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:52.950601 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6hc8\" (UniqueName: \"kubernetes.io/projected/44ea9c34-71b0-4e65-9730-9575d33bf414-kube-api-access-w6hc8\") pod \"44ea9c34-71b0-4e65-9730-9575d33bf414\" (UID: \"44ea9c34-71b0-4e65-9730-9575d33bf414\") " Jul 13 17:51:52.952869 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:52.952838 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44ea9c34-71b0-4e65-9730-9575d33bf414-kube-api-access-w6hc8" (OuterVolumeSpecName: "kube-api-access-w6hc8") pod "44ea9c34-71b0-4e65-9730-9575d33bf414" (UID: "44ea9c34-71b0-4e65-9730-9575d33bf414"). InnerVolumeSpecName "kube-api-access-w6hc8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:51:52.962189 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:52.962161 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44ea9c34-71b0-4e65-9730-9575d33bf414" path="/var/lib/kubelet/pods/44ea9c34-71b0-4e65-9730-9575d33bf414/volumes" Jul 13 17:51:53.016308 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:53.016276 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-8292-139830918242144"] Jul 13 17:51:53.016660 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:53.016645 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44ea9c34-71b0-4e65-9730-9575d33bf414" containerName="test-curl-8292-139830918241136" Jul 13 17:51:53.016660 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:53.016659 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="44ea9c34-71b0-4e65-9730-9575d33bf414" containerName="test-curl-8292-139830918241136" Jul 13 17:51:53.016767 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:53.016722 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="44ea9c34-71b0-4e65-9730-9575d33bf414" containerName="test-curl-8292-139830918241136" Jul 13 17:51:53.018952 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:53.018935 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830918242144" Jul 13 17:51:53.027579 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:53.027552 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-8292-139830918242144"] Jul 13 17:51:53.052429 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:53.052391 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6hc8\" (UniqueName: \"kubernetes.io/projected/44ea9c34-71b0-4e65-9730-9575d33bf414-kube-api-access-w6hc8\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:51:53.153797 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:53.153758 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhldr\" (UniqueName: \"kubernetes.io/projected/1073789f-921e-49dd-81f4-08535b5245df-kube-api-access-mhldr\") pod \"test-curl-8292-139830918242144\" (UID: \"1073789f-921e-49dd-81f4-08535b5245df\") " pod="openshift-ingress/test-curl-8292-139830918242144" Jul 13 17:51:53.255110 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:53.255019 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhldr\" (UniqueName: \"kubernetes.io/projected/1073789f-921e-49dd-81f4-08535b5245df-kube-api-access-mhldr\") pod \"test-curl-8292-139830918242144\" (UID: \"1073789f-921e-49dd-81f4-08535b5245df\") " pod="openshift-ingress/test-curl-8292-139830918242144" Jul 13 17:51:53.264575 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:53.264551 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhldr\" (UniqueName: \"kubernetes.io/projected/1073789f-921e-49dd-81f4-08535b5245df-kube-api-access-mhldr\") pod \"test-curl-8292-139830918242144\" (UID: \"1073789f-921e-49dd-81f4-08535b5245df\") " pod="openshift-ingress/test-curl-8292-139830918242144" Jul 13 17:51:53.329360 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:53.329315 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830918242144" Jul 13 17:51:53.471336 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:53.471283 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-8292-139830918242144"] Jul 13 17:51:53.694199 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:53.694171 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830918241136" Jul 13 17:51:53.694395 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:53.694222 2564 scope.go:117] "RemoveContainer" containerID="125856e7be08f946e9f95e2bc22147f1a94357d742c8350518bbf3b387d7bdb0" Jul 13 17:51:53.695642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:53.695615 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-8292-139830918242144" event={"ID":"1073789f-921e-49dd-81f4-08535b5245df","Type":"ContainerStarted","Data":"cacb053df9060bbc3e3743dd1e26932dd9cad915be40c636945bb9ea93f77c77"} Jul 13 17:51:54.702864 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:54.702821 2564 generic.go:356] "Generic (PLEG): container finished" podID="1073789f-921e-49dd-81f4-08535b5245df" containerID="fd2272b18920e442ef5c1ba08c3bfc9e9a7de8ac52a29c766c6d0dbce1aef4d7" exitCode=0 Jul 13 17:51:54.703319 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:54.702916 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-8292-139830918242144" event={"ID":"1073789f-921e-49dd-81f4-08535b5245df","Type":"ContainerDied","Data":"fd2272b18920e442ef5c1ba08c3bfc9e9a7de8ac52a29c766c6d0dbce1aef4d7"} Jul 13 17:51:55.857404 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:55.857383 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830918242144" Jul 13 17:51:55.878548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:55.878489 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-8292-139830918242144_1073789f-921e-49dd-81f4-08535b5245df/test-curl-8292-139830918242144/0.log" Jul 13 17:51:55.905783 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:55.905754 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-8292-139830918242144"] Jul 13 17:51:55.914619 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:55.914586 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-8292-139830918242144"] Jul 13 17:51:55.978285 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:55.978205 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhldr\" (UniqueName: \"kubernetes.io/projected/1073789f-921e-49dd-81f4-08535b5245df-kube-api-access-mhldr\") pod \"1073789f-921e-49dd-81f4-08535b5245df\" (UID: \"1073789f-921e-49dd-81f4-08535b5245df\") " Jul 13 17:51:55.980446 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:55.980422 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1073789f-921e-49dd-81f4-08535b5245df-kube-api-access-mhldr" (OuterVolumeSpecName: "kube-api-access-mhldr") pod "1073789f-921e-49dd-81f4-08535b5245df" (UID: "1073789f-921e-49dd-81f4-08535b5245df"). InnerVolumeSpecName "kube-api-access-mhldr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:51:56.042131 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:56.042101 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-8292-139830918242144"] Jul 13 17:51:56.042481 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:56.042462 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1073789f-921e-49dd-81f4-08535b5245df" containerName="test-curl-8292-139830918242144" Jul 13 17:51:56.042481 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:56.042478 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1073789f-921e-49dd-81f4-08535b5245df" containerName="test-curl-8292-139830918242144" Jul 13 17:51:56.042636 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:56.042561 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1073789f-921e-49dd-81f4-08535b5245df" containerName="test-curl-8292-139830918242144" Jul 13 17:51:56.044884 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:56.044865 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830918242144" Jul 13 17:51:56.052857 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:56.052828 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-8292-139830918242144"] Jul 13 17:51:56.079983 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:56.079952 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhldr\" (UniqueName: \"kubernetes.io/projected/1073789f-921e-49dd-81f4-08535b5245df-kube-api-access-mhldr\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:51:56.181123 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:56.181070 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lndx7\" (UniqueName: \"kubernetes.io/projected/dcd2a0ff-dc33-419d-a15f-1ba97f927e71-kube-api-access-lndx7\") pod \"test-curl-8292-139830918242144\" (UID: \"dcd2a0ff-dc33-419d-a15f-1ba97f927e71\") " pod="openshift-ingress/test-curl-8292-139830918242144" Jul 13 17:51:56.281777 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:56.281637 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lndx7\" (UniqueName: \"kubernetes.io/projected/dcd2a0ff-dc33-419d-a15f-1ba97f927e71-kube-api-access-lndx7\") pod \"test-curl-8292-139830918242144\" (UID: \"dcd2a0ff-dc33-419d-a15f-1ba97f927e71\") " pod="openshift-ingress/test-curl-8292-139830918242144" Jul 13 17:51:56.291817 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:56.291786 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lndx7\" (UniqueName: \"kubernetes.io/projected/dcd2a0ff-dc33-419d-a15f-1ba97f927e71-kube-api-access-lndx7\") pod \"test-curl-8292-139830918242144\" (UID: \"dcd2a0ff-dc33-419d-a15f-1ba97f927e71\") " pod="openshift-ingress/test-curl-8292-139830918242144" Jul 13 17:51:56.355307 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:56.355267 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830918242144" Jul 13 17:51:56.495663 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:56.495637 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-8292-139830918242144"] Jul 13 17:51:56.496943 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:51:56.496907 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddcd2a0ff_dc33_419d_a15f_1ba97f927e71.slice/crio-9b38cbf9ea18f2d0214498e6fe0d3a4a362335565ff66d103cdfc45206da6ede WatchSource:0}: Error finding container 9b38cbf9ea18f2d0214498e6fe0d3a4a362335565ff66d103cdfc45206da6ede: Status 404 returned error can't find the container with id 9b38cbf9ea18f2d0214498e6fe0d3a4a362335565ff66d103cdfc45206da6ede Jul 13 17:51:56.716688 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:56.716653 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830918242144" Jul 13 17:51:56.716688 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:56.716666 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cacb053df9060bbc3e3743dd1e26932dd9cad915be40c636945bb9ea93f77c77" Jul 13 17:51:56.717977 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:56.717950 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-8292-139830918242144" event={"ID":"dcd2a0ff-dc33-419d-a15f-1ba97f927e71","Type":"ContainerStarted","Data":"9b38cbf9ea18f2d0214498e6fe0d3a4a362335565ff66d103cdfc45206da6ede"} Jul 13 17:51:56.729502 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:56.729475 2564 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-8292-139830918242144" oldPodUID="1073789f-921e-49dd-81f4-08535b5245df" podUID="dcd2a0ff-dc33-419d-a15f-1ba97f927e71" Jul 13 17:51:56.960750 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:56.960708 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1073789f-921e-49dd-81f4-08535b5245df" path="/var/lib/kubelet/pods/1073789f-921e-49dd-81f4-08535b5245df/volumes" Jul 13 17:51:57.725054 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:57.725019 2564 generic.go:356] "Generic (PLEG): container finished" podID="dcd2a0ff-dc33-419d-a15f-1ba97f927e71" containerID="728fae93acb5762f52872297cae4e2942a3aa621eeec37a9adeccade4a1071b0" exitCode=0 Jul 13 17:51:57.725228 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:57.725067 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-8292-139830918242144" event={"ID":"dcd2a0ff-dc33-419d-a15f-1ba97f927e71","Type":"ContainerDied","Data":"728fae93acb5762f52872297cae4e2942a3aa621eeec37a9adeccade4a1071b0"} Jul 13 17:51:58.879916 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:58.879893 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830918242144" Jul 13 17:51:58.918022 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:58.917994 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-8292-139830918242144_dcd2a0ff-dc33-419d-a15f-1ba97f927e71/test-curl-8292-139830918242144/0.log" Jul 13 17:51:58.949698 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:58.949659 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-8292-139830918242144"] Jul 13 17:51:58.963564 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:58.963536 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-8292-139830918242144"] Jul 13 17:51:59.009365 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:59.009283 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lndx7\" (UniqueName: \"kubernetes.io/projected/dcd2a0ff-dc33-419d-a15f-1ba97f927e71-kube-api-access-lndx7\") pod \"dcd2a0ff-dc33-419d-a15f-1ba97f927e71\" (UID: \"dcd2a0ff-dc33-419d-a15f-1ba97f927e71\") " Jul 13 17:51:59.011525 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:59.011499 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcd2a0ff-dc33-419d-a15f-1ba97f927e71-kube-api-access-lndx7" (OuterVolumeSpecName: "kube-api-access-lndx7") pod "dcd2a0ff-dc33-419d-a15f-1ba97f927e71" (UID: "dcd2a0ff-dc33-419d-a15f-1ba97f927e71"). InnerVolumeSpecName "kube-api-access-lndx7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:51:59.110421 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:59.110386 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lndx7\" (UniqueName: \"kubernetes.io/projected/dcd2a0ff-dc33-419d-a15f-1ba97f927e71-kube-api-access-lndx7\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:51:59.739485 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:59.739448 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b38cbf9ea18f2d0214498e6fe0d3a4a362335565ff66d103cdfc45206da6ede" Jul 13 17:51:59.739485 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:59.739455 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-8292-139830918242144" Jul 13 17:51:59.750037 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:51:59.750000 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8292-139830918242144\" is forbidden: User \"system:node:ip-10-0-137-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-137-245.ec2.internal' and this object" podUID="dcd2a0ff-dc33-419d-a15f-1ba97f927e71" pod="openshift-ingress/test-curl-8292-139830918242144" Jul 13 17:51:59.876081 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:59.876043 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:51:59.979665 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:51:59.979627 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:52:00.961246 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:00.961210 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcd2a0ff-dc33-419d-a15f-1ba97f927e71" path="/var/lib/kubelet/pods/dcd2a0ff-dc33-419d-a15f-1ba97f927e71/volumes" Jul 13 17:52:09.776130 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:09.776084 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:52:13.160229 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:13.160190 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:52:14.278447 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:14.278403 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:52:20.819976 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:20.819936 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:52:20.921203 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:20.921165 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:52:25.044253 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:25.044213 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn"] Jul 13 17:52:25.044661 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:25.044452 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" podUID="59c7ae08-50d3-4dee-ad3d-ccebd837bd35" containerName="istio-proxy" containerID="cri-o://eb202907e196b4007bd0436de1d92601ef315280004cbdb2e3a3ba88c57ba2d4" gracePeriod=30 Jul 13 17:52:25.094171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:25.094124 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:52:25.283177 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:25.283138 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:52:26.120451 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:26.120410 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:52:26.221712 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:26.221677 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:52:29.098315 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:29.098272 2564 patch_prober.go:28] interesting pod/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 17:52:29.098699 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:29.098355 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" podUID="59c7ae08-50d3-4dee-ad3d-ccebd837bd35" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 17:52:30.302414 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.302390 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:52:30.378343 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.378252 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-workload-socket\") pod \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " Jul 13 17:52:30.378343 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.378304 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7sfs\" (UniqueName: \"kubernetes.io/projected/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-kube-api-access-n7sfs\") pod \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " Jul 13 17:52:30.378551 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.378352 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-podinfo\") pod \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " Jul 13 17:52:30.378551 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.378382 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-envoy\") pod \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " Jul 13 17:52:30.378551 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.378525 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-workload-certs\") pod \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " Jul 13 17:52:30.378710 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.378587 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istiod-ca-cert\") pod \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " Jul 13 17:52:30.378710 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.378630 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-data\") pod \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " Jul 13 17:52:30.378710 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.378654 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-token\") pod \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " Jul 13 17:52:30.378710 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.378582 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "59c7ae08-50d3-4dee-ad3d-ccebd837bd35" (UID: "59c7ae08-50d3-4dee-ad3d-ccebd837bd35"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:52:30.378710 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.378686 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-credential-socket\") pod \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\" (UID: \"59c7ae08-50d3-4dee-ad3d-ccebd837bd35\") " Jul 13 17:52:30.379004 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.378853 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "59c7ae08-50d3-4dee-ad3d-ccebd837bd35" (UID: "59c7ae08-50d3-4dee-ad3d-ccebd837bd35"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:52:30.379004 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.378943 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "59c7ae08-50d3-4dee-ad3d-ccebd837bd35" (UID: "59c7ae08-50d3-4dee-ad3d-ccebd837bd35"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:52:30.379104 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.379081 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "59c7ae08-50d3-4dee-ad3d-ccebd837bd35" (UID: "59c7ae08-50d3-4dee-ad3d-ccebd837bd35"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:52:30.379157 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.379144 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-workload-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:52:30.379211 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.379166 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-workload-certs\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:52:30.379211 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.379181 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istiod-ca-cert\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:52:30.380172 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.380147 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-data" (OuterVolumeSpecName: "istio-data") pod "59c7ae08-50d3-4dee-ad3d-ccebd837bd35" (UID: "59c7ae08-50d3-4dee-ad3d-ccebd837bd35"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:52:30.380919 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.380896 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-kube-api-access-n7sfs" (OuterVolumeSpecName: "kube-api-access-n7sfs") pod "59c7ae08-50d3-4dee-ad3d-ccebd837bd35" (UID: "59c7ae08-50d3-4dee-ad3d-ccebd837bd35"). InnerVolumeSpecName "kube-api-access-n7sfs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:52:30.381022 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.380980 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-token" (OuterVolumeSpecName: "istio-token") pod "59c7ae08-50d3-4dee-ad3d-ccebd837bd35" (UID: "59c7ae08-50d3-4dee-ad3d-ccebd837bd35"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:52:30.381170 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.381155 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "59c7ae08-50d3-4dee-ad3d-ccebd837bd35" (UID: "59c7ae08-50d3-4dee-ad3d-ccebd837bd35"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:52:30.381230 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.381176 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "59c7ae08-50d3-4dee-ad3d-ccebd837bd35" (UID: "59c7ae08-50d3-4dee-ad3d-ccebd837bd35"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:52:30.479634 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.479596 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-data\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:52:30.479634 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.479625 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-token\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:52:30.479634 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.479635 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-credential-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:52:30.479907 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.479645 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7sfs\" (UniqueName: \"kubernetes.io/projected/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-kube-api-access-n7sfs\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:52:30.479907 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.479656 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-podinfo\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:52:30.479907 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.479665 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59c7ae08-50d3-4dee-ad3d-ccebd837bd35-istio-envoy\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:52:30.918332 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.918293 2564 generic.go:356] "Generic (PLEG): container finished" podID="59c7ae08-50d3-4dee-ad3d-ccebd837bd35" containerID="eb202907e196b4007bd0436de1d92601ef315280004cbdb2e3a3ba88c57ba2d4" exitCode=0 Jul 13 17:52:30.918517 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.918344 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" Jul 13 17:52:30.918517 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.918378 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" event={"ID":"59c7ae08-50d3-4dee-ad3d-ccebd837bd35","Type":"ContainerDied","Data":"eb202907e196b4007bd0436de1d92601ef315280004cbdb2e3a3ba88c57ba2d4"} Jul 13 17:52:30.918517 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.918432 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn" event={"ID":"59c7ae08-50d3-4dee-ad3d-ccebd837bd35","Type":"ContainerDied","Data":"4ca45c864d0dd29049b08f28878ba55c80eaffce472742cada4e434f2ed9162e"} Jul 13 17:52:30.918517 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.918456 2564 scope.go:117] "RemoveContainer" containerID="eb202907e196b4007bd0436de1d92601ef315280004cbdb2e3a3ba88c57ba2d4" Jul 13 17:52:30.928464 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.928446 2564 scope.go:117] "RemoveContainer" containerID="748719dcf2d51699f918ea0b4c2e7bb238386b6e0503fe6f6757cd9139a34bcc" Jul 13 17:52:30.943658 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.943637 2564 scope.go:117] "RemoveContainer" containerID="eb202907e196b4007bd0436de1d92601ef315280004cbdb2e3a3ba88c57ba2d4" Jul 13 17:52:30.944029 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:52:30.944011 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb202907e196b4007bd0436de1d92601ef315280004cbdb2e3a3ba88c57ba2d4\": container with ID starting with eb202907e196b4007bd0436de1d92601ef315280004cbdb2e3a3ba88c57ba2d4 not found: ID does not exist" containerID="eb202907e196b4007bd0436de1d92601ef315280004cbdb2e3a3ba88c57ba2d4" Jul 13 17:52:30.944101 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.944038 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb202907e196b4007bd0436de1d92601ef315280004cbdb2e3a3ba88c57ba2d4"} err="failed to get container status \"eb202907e196b4007bd0436de1d92601ef315280004cbdb2e3a3ba88c57ba2d4\": rpc error: code = NotFound desc = could not find container \"eb202907e196b4007bd0436de1d92601ef315280004cbdb2e3a3ba88c57ba2d4\": container with ID starting with eb202907e196b4007bd0436de1d92601ef315280004cbdb2e3a3ba88c57ba2d4 not found: ID does not exist" Jul 13 17:52:30.944101 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.944057 2564 scope.go:117] "RemoveContainer" containerID="748719dcf2d51699f918ea0b4c2e7bb238386b6e0503fe6f6757cd9139a34bcc" Jul 13 17:52:30.944269 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:52:30.944250 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"748719dcf2d51699f918ea0b4c2e7bb238386b6e0503fe6f6757cd9139a34bcc\": container with ID starting with 748719dcf2d51699f918ea0b4c2e7bb238386b6e0503fe6f6757cd9139a34bcc not found: ID does not exist" containerID="748719dcf2d51699f918ea0b4c2e7bb238386b6e0503fe6f6757cd9139a34bcc" Jul 13 17:52:30.944309 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.944274 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"748719dcf2d51699f918ea0b4c2e7bb238386b6e0503fe6f6757cd9139a34bcc"} err="failed to get container status \"748719dcf2d51699f918ea0b4c2e7bb238386b6e0503fe6f6757cd9139a34bcc\": rpc error: code = NotFound desc = could not find container \"748719dcf2d51699f918ea0b4c2e7bb238386b6e0503fe6f6757cd9139a34bcc\": container with ID starting with 748719dcf2d51699f918ea0b4c2e7bb238386b6e0503fe6f6757cd9139a34bcc not found: ID does not exist" Jul 13 17:52:30.950653 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.950626 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn"] Jul 13 17:52:30.960430 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:30.960407 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-714452-openshift-default-5c7cdf5b5f-qvdbn"] Jul 13 17:52:32.961161 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:32.961127 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59c7ae08-50d3-4dee-ad3d-ccebd837bd35" path="/var/lib/kubelet/pods/59c7ae08-50d3-4dee-ad3d-ccebd837bd35/volumes" Jul 13 17:52:35.941347 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:35.941307 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:52:38.327082 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:38.327045 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9bmg9"] Jul 13 17:52:40.544127 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:40.544085 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:52:46.400377 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:46.400344 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:52:46.501879 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:46.501832 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:52:51.286278 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:51.286241 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h"] Jul 13 17:52:51.286722 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:51.286443 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" podUID="0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" containerName="istio-proxy" containerID="cri-o://5df2bed5ab966acfc0548bc995c80ef91307d4c2cb0572810255c6dd94e28dac" gracePeriod=30 Jul 13 17:52:51.305435 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:51.305406 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:52:51.531544 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:51.531501 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:52:56.540810 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.540779 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:52:56.712766 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.712707 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-podinfo\") pod \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " Jul 13 17:52:56.712983 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.712785 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-data\") pod \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " Jul 13 17:52:56.712983 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.712813 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-credential-socket\") pod \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " Jul 13 17:52:56.712983 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.712849 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istiod-ca-cert\") pod \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " Jul 13 17:52:56.712983 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.712890 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-workload-certs\") pod \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " Jul 13 17:52:56.712983 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.712941 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-token\") pod \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " Jul 13 17:52:56.712983 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.712967 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-envoy\") pod \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " Jul 13 17:52:56.713328 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.712998 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-workload-socket\") pod \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " Jul 13 17:52:56.713328 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.713032 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqhrl\" (UniqueName: \"kubernetes.io/projected/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-kube-api-access-jqhrl\") pod \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\" (UID: \"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff\") " Jul 13 17:52:56.713328 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.713158 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" (UID: "0ed40df3-56ea-4da2-a02e-05a5aa2b84ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:52:56.713328 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.713212 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" (UID: "0ed40df3-56ea-4da2-a02e-05a5aa2b84ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:52:56.713328 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.713298 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" (UID: "0ed40df3-56ea-4da2-a02e-05a5aa2b84ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:52:56.713596 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.713508 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" (UID: "0ed40df3-56ea-4da2-a02e-05a5aa2b84ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:52:56.713596 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.713420 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-credential-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:52:56.713596 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.713547 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istiod-ca-cert\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:52:56.713596 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.713562 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-workload-certs\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:52:56.714007 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.713870 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-data" (OuterVolumeSpecName: "istio-data") pod "0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" (UID: "0ed40df3-56ea-4da2-a02e-05a5aa2b84ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:52:56.715325 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.715288 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-token" (OuterVolumeSpecName: "istio-token") pod "0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" (UID: "0ed40df3-56ea-4da2-a02e-05a5aa2b84ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:52:56.715325 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.715301 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" (UID: "0ed40df3-56ea-4da2-a02e-05a5aa2b84ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:52:56.715467 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.715320 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" (UID: "0ed40df3-56ea-4da2-a02e-05a5aa2b84ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:52:56.715467 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.715439 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-kube-api-access-jqhrl" (OuterVolumeSpecName: "kube-api-access-jqhrl") pod "0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" (UID: "0ed40df3-56ea-4da2-a02e-05a5aa2b84ff"). InnerVolumeSpecName "kube-api-access-jqhrl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:52:56.814007 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.813971 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-token\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:52:56.814007 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.814002 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-envoy\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:52:56.814007 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.814011 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-workload-socket\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:52:56.814289 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.814022 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqhrl\" (UniqueName: \"kubernetes.io/projected/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-kube-api-access-jqhrl\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:52:56.814289 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.814032 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-podinfo\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:52:56.814289 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:56.814043 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff-istio-data\") on node \"ip-10-0-137-245.ec2.internal\" DevicePath \"\"" Jul 13 17:52:57.062425 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:57.062388 2564 generic.go:356] "Generic (PLEG): container finished" podID="0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" containerID="5df2bed5ab966acfc0548bc995c80ef91307d4c2cb0572810255c6dd94e28dac" exitCode=0 Jul 13 17:52:57.062639 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:57.062439 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" Jul 13 17:52:57.062639 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:57.062482 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" event={"ID":"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff","Type":"ContainerDied","Data":"5df2bed5ab966acfc0548bc995c80ef91307d4c2cb0572810255c6dd94e28dac"} Jul 13 17:52:57.062639 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:57.062549 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h" event={"ID":"0ed40df3-56ea-4da2-a02e-05a5aa2b84ff","Type":"ContainerDied","Data":"2fd28ed3d4c4176825e6dbb8fe7f0237710dd7b814efd8042f270e8d5f77f8bf"} Jul 13 17:52:57.062639 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:57.062581 2564 scope.go:117] "RemoveContainer" containerID="5df2bed5ab966acfc0548bc995c80ef91307d4c2cb0572810255c6dd94e28dac" Jul 13 17:52:57.072568 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:57.072523 2564 scope.go:117] "RemoveContainer" containerID="b585300960bd70e9ad595f703879ab0b05835559b5c1be7ab54ddeeb54e1cc52" Jul 13 17:52:57.084298 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:57.084234 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-l5n48_430f9856-48e6-46e5-b4e2-83ce4fcf0e98/manager/0.log" Jul 13 17:52:57.084685 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:57.084617 2564 scope.go:117] "RemoveContainer" containerID="5df2bed5ab966acfc0548bc995c80ef91307d4c2cb0572810255c6dd94e28dac" Jul 13 17:52:57.085120 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:52:57.085090 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5df2bed5ab966acfc0548bc995c80ef91307d4c2cb0572810255c6dd94e28dac\": container with ID starting with 5df2bed5ab966acfc0548bc995c80ef91307d4c2cb0572810255c6dd94e28dac not found: ID does not exist" containerID="5df2bed5ab966acfc0548bc995c80ef91307d4c2cb0572810255c6dd94e28dac" Jul 13 17:52:57.085218 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:57.085131 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5df2bed5ab966acfc0548bc995c80ef91307d4c2cb0572810255c6dd94e28dac"} err="failed to get container status \"5df2bed5ab966acfc0548bc995c80ef91307d4c2cb0572810255c6dd94e28dac\": rpc error: code = NotFound desc = could not find container \"5df2bed5ab966acfc0548bc995c80ef91307d4c2cb0572810255c6dd94e28dac\": container with ID starting with 5df2bed5ab966acfc0548bc995c80ef91307d4c2cb0572810255c6dd94e28dac not found: ID does not exist" Jul 13 17:52:57.085218 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:57.085156 2564 scope.go:117] "RemoveContainer" containerID="b585300960bd70e9ad595f703879ab0b05835559b5c1be7ab54ddeeb54e1cc52" Jul 13 17:52:57.085454 ip-10-0-137-245 kubenswrapper[2564]: E0713 17:52:57.085433 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b585300960bd70e9ad595f703879ab0b05835559b5c1be7ab54ddeeb54e1cc52\": container with ID starting with b585300960bd70e9ad595f703879ab0b05835559b5c1be7ab54ddeeb54e1cc52 not found: ID does not exist" containerID="b585300960bd70e9ad595f703879ab0b05835559b5c1be7ab54ddeeb54e1cc52" Jul 13 17:52:57.085540 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:57.085464 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b585300960bd70e9ad595f703879ab0b05835559b5c1be7ab54ddeeb54e1cc52"} err="failed to get container status \"b585300960bd70e9ad595f703879ab0b05835559b5c1be7ab54ddeeb54e1cc52\": rpc error: code = NotFound desc = could not find container \"b585300960bd70e9ad595f703879ab0b05835559b5c1be7ab54ddeeb54e1cc52\": container with ID starting with b585300960bd70e9ad595f703879ab0b05835559b5c1be7ab54ddeeb54e1cc52 not found: ID does not exist" Jul 13 17:52:57.086087 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:57.086072 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h"] Jul 13 17:52:57.088393 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:57.088374 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-31a784-openshift-default-56c6f95f4c-hhf8h"] Jul 13 17:52:57.215190 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:57.215153 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7nvzq_ec8ec3a3-9414-48ce-bb17-2344ec6286b8/manager/0.log" Jul 13 17:52:57.474804 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:57.474770 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7hstg_d7042803-711a-4efb-a0ac-5a1825bc5d7d/server/0.log" Jul 13 17:52:57.600171 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:57.600144 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-9j2b6_3aec7a8a-c182-4055-8818-e23a9e1645d0/manager/0.log" Jul 13 17:52:57.720460 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:57.720434 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dfb6c5f74-lzk54_c09bd7aa-9223-4fed-bd75-2578ec1605b3/copy-manifests/0.log" Jul 13 17:52:57.732576 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:57.732508 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dfb6c5f74-lzk54_c09bd7aa-9223-4fed-bd75-2578ec1605b3/manager/0.log" Jul 13 17:52:58.497232 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:58.497200 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29732730-m66p9_2ecb4eb4-0272-46e1-8020-fc2c220122e8/cleanup/0.log" Jul 13 17:52:58.716179 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:58.716148 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-0e7283f7-kp787_0eea6d6e-c248-460f-a3d8-447bbd419ebb/revoke-keys/0.log" Jul 13 17:52:58.830257 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:58.830175 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-85580aa7-glfsk_e0f2f067-d17c-4ac1-bd10-a83de91c3d82/revoke-keys/0.log" Jul 13 17:52:58.960978 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:58.960941 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" path="/var/lib/kubelet/pods/0ed40df3-56ea-4da2-a02e-05a5aa2b84ff/volumes" Jul 13 17:52:59.176637 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:59.176607 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-8e6203f1-gjdks_60221c77-ad6d-4c47-b436-f9d84879e2be/revoke-keys/0.log" Jul 13 17:52:59.289540 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:59.289511 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-6a3680-czrtg_c3d77ff2-c57b-48ff-a57b-cb8c1ccdaebe/revoke-keys/0.log" Jul 13 17:52:59.548459 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:59.548380 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-b8ebfd6b-zgnzg_06511053-9573-478e-a627-56c78c0b876c/revoke-keys/0.log" Jul 13 17:52:59.660473 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:52:59.660446 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-f4b51184-f789r_7f949327-fe89-4565-9610-1d2a9c9d2b4a/revoke-keys/0.log" Jul 13 17:53:00.857509 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:00.857481 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb_3236fc6b-d795-48f4-8298-2866758c75b5/util/0.log" Jul 13 17:53:00.868972 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:00.868939 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb_3236fc6b-d795-48f4-8298-2866758c75b5/pull/0.log" Jul 13 17:53:00.876465 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:00.876435 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb_3236fc6b-d795-48f4-8298-2866758c75b5/extract/0.log" Jul 13 17:53:00.991781 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:00.991753 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx_99f3124a-f8f6-403b-9984-b4aacf9e5e74/util/0.log" Jul 13 17:53:01.005781 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:01.005754 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx_99f3124a-f8f6-403b-9984-b4aacf9e5e74/pull/0.log" Jul 13 17:53:01.013385 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:01.013362 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx_99f3124a-f8f6-403b-9984-b4aacf9e5e74/extract/0.log" Jul 13 17:53:01.128187 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:01.128088 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2_2e440e1a-c3c6-4903-88d5-fc70b40a4c17/util/0.log" Jul 13 17:53:01.135971 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:01.135942 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2_2e440e1a-c3c6-4903-88d5-fc70b40a4c17/pull/0.log" Jul 13 17:53:01.145225 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:01.145198 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2_2e440e1a-c3c6-4903-88d5-fc70b40a4c17/extract/0.log" Jul 13 17:53:01.264916 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:01.264876 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs_9d3aec72-80db-4422-a290-45527c4d9c6d/util/0.log" Jul 13 17:53:01.272377 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:01.272337 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs_9d3aec72-80db-4422-a290-45527c4d9c6d/pull/0.log" Jul 13 17:53:01.279688 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:01.279658 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs_9d3aec72-80db-4422-a290-45527c4d9c6d/extract/0.log" Jul 13 17:53:02.043170 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:02.043140 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-6mcm8_ee31e5fa-ca47-4680-a3b4-a7c517ad0506/manager/0.log" Jul 13 17:53:02.164525 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:02.164494 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-9bmg9_709376d2-4a59-46e5-a1a1-987b0a525e95/limitador/0.log" Jul 13 17:53:02.648389 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:02.648351 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf449pjx_c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13/istio-proxy/0.log" Jul 13 17:53:02.769255 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:02.769225 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-cf8tz_a269ed30-312c-421b-aa7e-d45f32aab007/discovery/0.log" Jul 13 17:53:02.890005 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:02.889970 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-746dfd5849-cm44k_098a1fa8-7634-46cf-a499-5d6b3c627fb6/kube-auth-proxy/0.log" Jul 13 17:53:03.129837 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:03.129742 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-84jgb_6db06f8c-f4ec-49d9-94c1-9dade34c33f8/istio-proxy/1.log" Jul 13 17:53:08.366526 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.366482 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rq8n6/must-gather-szhc5"] Jul 13 17:53:08.367030 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.367001 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59c7ae08-50d3-4dee-ad3d-ccebd837bd35" containerName="istio-proxy" Jul 13 17:53:08.367030 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.367020 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="59c7ae08-50d3-4dee-ad3d-ccebd837bd35" containerName="istio-proxy" Jul 13 17:53:08.367156 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.367036 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcd2a0ff-dc33-419d-a15f-1ba97f927e71" containerName="test-curl-8292-139830918242144" Jul 13 17:53:08.367156 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.367046 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcd2a0ff-dc33-419d-a15f-1ba97f927e71" containerName="test-curl-8292-139830918242144" Jul 13 17:53:08.367156 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.367063 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59c7ae08-50d3-4dee-ad3d-ccebd837bd35" containerName="istio-proxy" Jul 13 17:53:08.367156 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.367072 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="59c7ae08-50d3-4dee-ad3d-ccebd837bd35" containerName="istio-proxy" Jul 13 17:53:08.367156 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.367085 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" containerName="istio-proxy" Jul 13 17:53:08.367156 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.367095 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" containerName="istio-proxy" Jul 13 17:53:08.367156 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.367116 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" containerName="istio-proxy" Jul 13 17:53:08.367156 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.367124 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" containerName="istio-proxy" Jul 13 17:53:08.367536 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.367207 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" containerName="istio-proxy" Jul 13 17:53:08.367536 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.367221 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ed40df3-56ea-4da2-a02e-05a5aa2b84ff" containerName="istio-proxy" Jul 13 17:53:08.367536 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.367232 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="59c7ae08-50d3-4dee-ad3d-ccebd837bd35" containerName="istio-proxy" Jul 13 17:53:08.367536 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.367244 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcd2a0ff-dc33-419d-a15f-1ba97f927e71" containerName="test-curl-8292-139830918242144" Jul 13 17:53:08.367536 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.367254 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="59c7ae08-50d3-4dee-ad3d-ccebd837bd35" containerName="istio-proxy" Jul 13 17:53:08.370583 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.370559 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rq8n6/must-gather-szhc5" Jul 13 17:53:08.373685 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.373605 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rq8n6\"/\"openshift-service-ca.crt\"" Jul 13 17:53:08.374008 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.373864 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rq8n6\"/\"default-dockercfg-2c2l5\"" Jul 13 17:53:08.375158 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.375132 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rq8n6\"/\"kube-root-ca.crt\"" Jul 13 17:53:08.378657 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.378636 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rq8n6/must-gather-szhc5"] Jul 13 17:53:08.379393 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.379355 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:53:08.408122 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.408085 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgrr8\" (UniqueName: \"kubernetes.io/projected/3977a5e0-28cf-4d6e-b8b3-29288ced5bd2-kube-api-access-tgrr8\") pod \"must-gather-szhc5\" (UID: \"3977a5e0-28cf-4d6e-b8b3-29288ced5bd2\") " pod="openshift-must-gather-rq8n6/must-gather-szhc5" Jul 13 17:53:08.408122 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.408124 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3977a5e0-28cf-4d6e-b8b3-29288ced5bd2-must-gather-output\") pod \"must-gather-szhc5\" (UID: \"3977a5e0-28cf-4d6e-b8b3-29288ced5bd2\") " pod="openshift-must-gather-rq8n6/must-gather-szhc5" Jul 13 17:53:08.475506 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.475473 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:53:08.509183 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.509151 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgrr8\" (UniqueName: \"kubernetes.io/projected/3977a5e0-28cf-4d6e-b8b3-29288ced5bd2-kube-api-access-tgrr8\") pod \"must-gather-szhc5\" (UID: \"3977a5e0-28cf-4d6e-b8b3-29288ced5bd2\") " pod="openshift-must-gather-rq8n6/must-gather-szhc5" Jul 13 17:53:08.509337 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.509188 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3977a5e0-28cf-4d6e-b8b3-29288ced5bd2-must-gather-output\") pod \"must-gather-szhc5\" (UID: \"3977a5e0-28cf-4d6e-b8b3-29288ced5bd2\") " pod="openshift-must-gather-rq8n6/must-gather-szhc5" Jul 13 17:53:08.509533 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.509517 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3977a5e0-28cf-4d6e-b8b3-29288ced5bd2-must-gather-output\") pod \"must-gather-szhc5\" (UID: \"3977a5e0-28cf-4d6e-b8b3-29288ced5bd2\") " pod="openshift-must-gather-rq8n6/must-gather-szhc5" Jul 13 17:53:08.517614 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.517583 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgrr8\" (UniqueName: \"kubernetes.io/projected/3977a5e0-28cf-4d6e-b8b3-29288ced5bd2-kube-api-access-tgrr8\") pod \"must-gather-szhc5\" (UID: \"3977a5e0-28cf-4d6e-b8b3-29288ced5bd2\") " pod="openshift-must-gather-rq8n6/must-gather-szhc5" Jul 13 17:53:08.681903 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.681865 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rq8n6/must-gather-szhc5" Jul 13 17:53:08.826828 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:08.826804 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rq8n6/must-gather-szhc5"] Jul 13 17:53:08.827594 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:53:08.827562 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3977a5e0_28cf_4d6e_b8b3_29288ced5bd2.slice/crio-52bc399f9c8279f02714a07681ce4a57dbc03c3d2507796fbc01dc6ba15710d9 WatchSource:0}: Error finding container 52bc399f9c8279f02714a07681ce4a57dbc03c3d2507796fbc01dc6ba15710d9: Status 404 returned error can't find the container with id 52bc399f9c8279f02714a07681ce4a57dbc03c3d2507796fbc01dc6ba15710d9 Jul 13 17:53:09.141984 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:09.141889 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rq8n6/must-gather-szhc5" event={"ID":"3977a5e0-28cf-4d6e-b8b3-29288ced5bd2","Type":"ContainerStarted","Data":"52bc399f9c8279f02714a07681ce4a57dbc03c3d2507796fbc01dc6ba15710d9"} Jul 13 17:53:09.349350 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:09.349312 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:53:10.149538 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:10.149500 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rq8n6/must-gather-szhc5" event={"ID":"3977a5e0-28cf-4d6e-b8b3-29288ced5bd2","Type":"ContainerStarted","Data":"f9a590783599dcd26fd3998ffbccc68c0df410f4f9ed4e733ba6ee5d2cd5a081"} Jul 13 17:53:10.149942 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:10.149551 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rq8n6/must-gather-szhc5" event={"ID":"3977a5e0-28cf-4d6e-b8b3-29288ced5bd2","Type":"ContainerStarted","Data":"77b03d820bfaa6d8fadbdffcc95223219209a8a523ca226f1d90e5d38942800c"} Jul 13 17:53:10.167950 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:10.167895 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rq8n6/must-gather-szhc5" podStartSLOduration=1.053512989 podStartE2EDuration="2.167877761s" totalImagesPullingTime="1.114364772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:53:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:53:10.164960743 +0000 UTC m=+2867.827735932" watchObservedRunningTime="2026-07-13 17:53:10.167877761 +0000 UTC m=+2867.830652950" Jul 13 17:53:11.533485 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:11.533455 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kj2dr_8dcab0f9-5abb-4fa7-af28-54be545eb542/global-pull-secret-syncer/0.log" Jul 13 17:53:11.668197 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:11.668163 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" Jul 13 17:53:11.780447 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:11.780413 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-245.ec2.internal_abc7e90015ad279ad8efd2a082fbdc6c/haproxy/0.log" Jul 13 17:53:16.218006 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:16.217918 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb_3236fc6b-d795-48f4-8298-2866758c75b5/extract/0.log" Jul 13 17:53:16.240305 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:16.240270 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb_3236fc6b-d795-48f4-8298-2866758c75b5/util/0.log" Jul 13 17:53:16.264968 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:16.264935 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tflrb_3236fc6b-d795-48f4-8298-2866758c75b5/pull/0.log" Jul 13 17:53:16.297654 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:16.297620 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx_99f3124a-f8f6-403b-9984-b4aacf9e5e74/extract/0.log" Jul 13 17:53:16.329859 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:16.329811 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx_99f3124a-f8f6-403b-9984-b4aacf9e5e74/util/0.log" Jul 13 17:53:16.353161 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:16.353068 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04p7mx_99f3124a-f8f6-403b-9984-b4aacf9e5e74/pull/0.log" Jul 13 17:53:16.409147 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:16.406054 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2_2e440e1a-c3c6-4903-88d5-fc70b40a4c17/extract/0.log" Jul 13 17:53:16.427558 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:16.427490 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2_2e440e1a-c3c6-4903-88d5-fc70b40a4c17/util/0.log" Jul 13 17:53:16.449563 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:16.449519 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lgl2_2e440e1a-c3c6-4903-88d5-fc70b40a4c17/pull/0.log" Jul 13 17:53:16.478237 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:16.478163 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs_9d3aec72-80db-4422-a290-45527c4d9c6d/extract/0.log" Jul 13 17:53:16.500116 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:16.500066 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs_9d3aec72-80db-4422-a290-45527c4d9c6d/util/0.log" Jul 13 17:53:16.522671 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:16.522634 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g6xvs_9d3aec72-80db-4422-a290-45527c4d9c6d/pull/0.log" Jul 13 17:53:17.107756 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:17.104134 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-6mcm8_ee31e5fa-ca47-4680-a3b4-a7c517ad0506/manager/0.log" Jul 13 17:53:17.142514 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:17.142481 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-9bmg9_709376d2-4a59-46e5-a1a1-987b0a525e95/limitador/0.log" Jul 13 17:53:19.264560 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:19.264530 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mcn2q_83ae6253-ea82-42e8-9ae5-2fa29ada5204/node-exporter/0.log" Jul 13 17:53:19.287988 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:19.287945 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mcn2q_83ae6253-ea82-42e8-9ae5-2fa29ada5204/kube-rbac-proxy/0.log" Jul 13 17:53:19.315763 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:19.315697 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mcn2q_83ae6253-ea82-42e8-9ae5-2fa29ada5204/init-textfile/0.log" Jul 13 17:53:19.704642 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:19.704609 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-zqrh4_d2341d47-022a-4f3f-bf9e-b8abb0caf1af/prometheus-operator/0.log" Jul 13 17:53:19.723547 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:19.723505 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-zqrh4_d2341d47-022a-4f3f-bf9e-b8abb0caf1af/kube-rbac-proxy/0.log" Jul 13 17:53:19.751818 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:19.751782 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-b7ptl_d384e72f-14f4-4dae-a011-1c78c2700765/prometheus-operator-admission-webhook/0.log" Jul 13 17:53:19.789838 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:19.789806 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76dc565d9f-w97hl_57625ec7-ea64-4acd-a884-68fec13cadbe/telemeter-client/0.log" Jul 13 17:53:19.813621 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:19.813587 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76dc565d9f-w97hl_57625ec7-ea64-4acd-a884-68fec13cadbe/reload/0.log" Jul 13 17:53:19.840111 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:19.840086 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76dc565d9f-w97hl_57625ec7-ea64-4acd-a884-68fec13cadbe/kube-rbac-proxy/0.log" Jul 13 17:53:21.781068 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:21.781023 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m"] Jul 13 17:53:21.785383 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:21.785355 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:21.794249 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:21.794170 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m"] Jul 13 17:53:21.845594 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:21.845558 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f4d17c49-78a7-4919-b06f-8d99b43d7b40-proc\") pod \"perf-node-gather-daemonset-jmq5m\" (UID: \"f4d17c49-78a7-4919-b06f-8d99b43d7b40\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:21.845809 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:21.845625 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f4d17c49-78a7-4919-b06f-8d99b43d7b40-podres\") pod \"perf-node-gather-daemonset-jmq5m\" (UID: \"f4d17c49-78a7-4919-b06f-8d99b43d7b40\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:21.845809 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:21.845671 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4d17c49-78a7-4919-b06f-8d99b43d7b40-lib-modules\") pod \"perf-node-gather-daemonset-jmq5m\" (UID: \"f4d17c49-78a7-4919-b06f-8d99b43d7b40\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:21.845809 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:21.845692 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c74sc\" (UniqueName: \"kubernetes.io/projected/f4d17c49-78a7-4919-b06f-8d99b43d7b40-kube-api-access-c74sc\") pod \"perf-node-gather-daemonset-jmq5m\" (UID: \"f4d17c49-78a7-4919-b06f-8d99b43d7b40\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:21.845809 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:21.845784 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4d17c49-78a7-4919-b06f-8d99b43d7b40-sys\") pod \"perf-node-gather-daemonset-jmq5m\" (UID: \"f4d17c49-78a7-4919-b06f-8d99b43d7b40\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:21.946531 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:21.946485 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4d17c49-78a7-4919-b06f-8d99b43d7b40-sys\") pod \"perf-node-gather-daemonset-jmq5m\" (UID: \"f4d17c49-78a7-4919-b06f-8d99b43d7b40\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:21.946783 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:21.946759 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4d17c49-78a7-4919-b06f-8d99b43d7b40-sys\") pod \"perf-node-gather-daemonset-jmq5m\" (UID: \"f4d17c49-78a7-4919-b06f-8d99b43d7b40\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:21.947010 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:21.946991 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f4d17c49-78a7-4919-b06f-8d99b43d7b40-proc\") pod \"perf-node-gather-daemonset-jmq5m\" (UID: \"f4d17c49-78a7-4919-b06f-8d99b43d7b40\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:21.947149 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:21.947130 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f4d17c49-78a7-4919-b06f-8d99b43d7b40-proc\") pod \"perf-node-gather-daemonset-jmq5m\" (UID: \"f4d17c49-78a7-4919-b06f-8d99b43d7b40\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:21.947266 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:21.947182 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f4d17c49-78a7-4919-b06f-8d99b43d7b40-podres\") pod \"perf-node-gather-daemonset-jmq5m\" (UID: \"f4d17c49-78a7-4919-b06f-8d99b43d7b40\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:21.947418 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:21.947402 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4d17c49-78a7-4919-b06f-8d99b43d7b40-lib-modules\") pod \"perf-node-gather-daemonset-jmq5m\" (UID: \"f4d17c49-78a7-4919-b06f-8d99b43d7b40\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:21.947550 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:21.947534 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c74sc\" (UniqueName: \"kubernetes.io/projected/f4d17c49-78a7-4919-b06f-8d99b43d7b40-kube-api-access-c74sc\") pod \"perf-node-gather-daemonset-jmq5m\" (UID: \"f4d17c49-78a7-4919-b06f-8d99b43d7b40\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:21.947690 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:21.947545 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4d17c49-78a7-4919-b06f-8d99b43d7b40-lib-modules\") pod \"perf-node-gather-daemonset-jmq5m\" (UID: \"f4d17c49-78a7-4919-b06f-8d99b43d7b40\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:21.947905 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:21.947309 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f4d17c49-78a7-4919-b06f-8d99b43d7b40-podres\") pod \"perf-node-gather-daemonset-jmq5m\" (UID: \"f4d17c49-78a7-4919-b06f-8d99b43d7b40\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:21.957135 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:21.957081 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c74sc\" (UniqueName: \"kubernetes.io/projected/f4d17c49-78a7-4919-b06f-8d99b43d7b40-kube-api-access-c74sc\") pod \"perf-node-gather-daemonset-jmq5m\" (UID: \"f4d17c49-78a7-4919-b06f-8d99b43d7b40\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:22.101408 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:22.101322 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:22.306431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:22.306389 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m"] Jul 13 17:53:22.308498 ip-10-0-137-245 kubenswrapper[2564]: W0713 17:53:22.308445 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf4d17c49_78a7_4919_b06f_8d99b43d7b40.slice/crio-16571c4d5dfd8b0cc0e1ad569c2cede74f4c2cc1838b270682fee62d783a6d05 WatchSource:0}: Error finding container 16571c4d5dfd8b0cc0e1ad569c2cede74f4c2cc1838b270682fee62d783a6d05: Status 404 returned error can't find the container with id 16571c4d5dfd8b0cc0e1ad569c2cede74f4c2cc1838b270682fee62d783a6d05 Jul 13 17:53:22.451638 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:22.451601 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5978bd6596-pgchn_da10a14e-1b38-40a1-b736-4c20b1862f71/console/0.log" Jul 13 17:53:22.494515 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:22.494476 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-pmzkl_47c45b3d-1255-4dc7-ab98-97a443293a83/download-server/0.log" Jul 13 17:53:23.269744 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:23.269675 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" event={"ID":"f4d17c49-78a7-4919-b06f-8d99b43d7b40","Type":"ContainerStarted","Data":"b5461f3d419416fcd99662bd253385ed22ee8dfa9d80cea508bdc243842cde66"} Jul 13 17:53:23.270235 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:23.269770 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" event={"ID":"f4d17c49-78a7-4919-b06f-8d99b43d7b40","Type":"ContainerStarted","Data":"16571c4d5dfd8b0cc0e1ad569c2cede74f4c2cc1838b270682fee62d783a6d05"} Jul 13 17:53:23.270235 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:23.269900 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:23.288798 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:23.288704 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" podStartSLOduration=2.288682004 podStartE2EDuration="2.288682004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:53:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:53:23.286905875 +0000 UTC m=+2880.949681063" watchObservedRunningTime="2026-07-13 17:53:23.288682004 +0000 UTC m=+2880.951457194" Jul 13 17:53:24.043039 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:24.043012 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-64h5v_3a8b92b2-17ca-4115-a0a1-6410f7fafb6a/dns/0.log" Jul 13 17:53:24.062665 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:24.062633 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-64h5v_3a8b92b2-17ca-4115-a0a1-6410f7fafb6a/kube-rbac-proxy/0.log" Jul 13 17:53:24.209048 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:24.209015 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bm8v7_33e43fe0-fc83-41c0-8a5c-9cdf168c0d9d/dns-node-resolver/0.log" Jul 13 17:53:24.751719 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:24.751678 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-678db6845d-xhfqs_3c5e9f64-15e5-484e-a78d-4f47e77f25d6/registry/0.log" Jul 13 17:53:24.794620 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:24.794593 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vq8xq_ab1216cc-4528-474c-a329-daefce87939c/node-ca/0.log" Jul 13 17:53:25.728177 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:25.728146 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf449pjx_c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13/istio-proxy/0.log" Jul 13 17:53:25.864677 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:25.864646 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-cf8tz_a269ed30-312c-421b-aa7e-d45f32aab007/discovery/0.log" Jul 13 17:53:25.895910 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:25.895874 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-746dfd5849-cm44k_098a1fa8-7634-46cf-a499-5d6b3c627fb6/kube-auth-proxy/0.log" Jul 13 17:53:25.944134 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:25.944095 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-84jgb_6db06f8c-f4ec-49d9-94c1-9dade34c33f8/istio-proxy/0.log" Jul 13 17:53:25.982501 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:25.982399 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-84jgb_6db06f8c-f4ec-49d9-94c1-9dade34c33f8/istio-proxy/1.log" Jul 13 17:53:26.751711 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:26.751680 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jjsrf_9e4de773-1f52-44a9-9077-1ae31610835b/serve-healthcheck-canary/0.log" Jul 13 17:53:27.381180 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:27.381148 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qqmx6_0807a42f-0465-4db7-8ce1-880139f0c35f/kube-rbac-proxy/0.log" Jul 13 17:53:27.400709 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:27.400676 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qqmx6_0807a42f-0465-4db7-8ce1-880139f0c35f/exporter/0.log" Jul 13 17:53:27.421242 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:27.421211 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qqmx6_0807a42f-0465-4db7-8ce1-880139f0c35f/extractor/0.log" Jul 13 17:53:29.289612 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:29.289578 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-jmq5m" Jul 13 17:53:29.521431 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:29.521403 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-l5n48_430f9856-48e6-46e5-b4e2-83ce4fcf0e98/manager/0.log" Jul 13 17:53:29.690998 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:29.690969 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7nvzq_ec8ec3a3-9414-48ce-bb17-2344ec6286b8/manager/0.log" Jul 13 17:53:29.824565 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:29.824529 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7hstg_d7042803-711a-4efb-a0ac-5a1825bc5d7d/server/0.log" Jul 13 17:53:29.914175 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:29.914139 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-9j2b6_3aec7a8a-c182-4055-8818-e23a9e1645d0/manager/0.log" Jul 13 17:53:30.002000 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:30.001913 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dfb6c5f74-lzk54_c09bd7aa-9223-4fed-bd75-2578ec1605b3/manager/0.log" Jul 13 17:53:30.021075 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:30.021031 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dfb6c5f74-lzk54_c09bd7aa-9223-4fed-bd75-2578ec1605b3/copy-manifests/0.log" Jul 13 17:53:32.297324 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:32.297236 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf449pjx_c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13/istio-proxy/0.log" Jul 13 17:53:32.426548 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:32.426513 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-cf8tz_a269ed30-312c-421b-aa7e-d45f32aab007/discovery/0.log" Jul 13 17:53:32.436951 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:32.436924 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-746dfd5849-cm44k_098a1fa8-7634-46cf-a499-5d6b3c627fb6/kube-auth-proxy/0.log" Jul 13 17:53:32.468493 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:32.468420 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-84jgb_6db06f8c-f4ec-49d9-94c1-9dade34c33f8/istio-proxy/0.log" Jul 13 17:53:32.516471 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:32.516440 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-84jgb_6db06f8c-f4ec-49d9-94c1-9dade34c33f8/istio-proxy/1.log" Jul 13 17:53:33.804484 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:33.804428 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf449pjx_c52307b7-2e8c-46b0-90bb-8ff1ba9e0c13/istio-proxy/0.log" Jul 13 17:53:33.942231 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:33.942197 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-cf8tz_a269ed30-312c-421b-aa7e-d45f32aab007/discovery/0.log" Jul 13 17:53:33.955261 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:33.955234 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-746dfd5849-cm44k_098a1fa8-7634-46cf-a499-5d6b3c627fb6/kube-auth-proxy/0.log" Jul 13 17:53:33.999827 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:33.999680 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-84jgb_6db06f8c-f4ec-49d9-94c1-9dade34c33f8/istio-proxy/0.log" Jul 13 17:53:34.042411 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:34.042371 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-84jgb_6db06f8c-f4ec-49d9-94c1-9dade34c33f8/istio-proxy/1.log" Jul 13 17:53:36.693635 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:36.693602 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c8xq8_71c06287-e098-4864-9a00-fc294a063aa0/konnectivity-agent/0.log" limit=1024 Jul 13 17:53:38.017693 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:38.017659 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4fwz6_c891937a-13fe-41d6-b824-2d569278dedb/kube-multus/0.log" Jul 13 17:53:38.219277 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:38.219249 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc8ng_e6411f69-741b-44f2-ae0f-7babdd39a621/kube-multus-additional-cni-plugins/0.log" Jul 13 17:53:38.246156 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:38.246127 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc8ng_e6411f69-741b-44f2-ae0f-7babdd39a621/egress-router-binary-copy/0.log" Jul 13 17:53:38.269222 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:38.269133 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc8ng_e6411f69-741b-44f2-ae0f-7babdd39a621/cni-plugins/0.log" Jul 13 17:53:38.289798 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:38.289762 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc8ng_e6411f69-741b-44f2-ae0f-7babdd39a621/bond-cni-plugin/0.log" Jul 13 17:53:38.309980 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:38.309944 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc8ng_e6411f69-741b-44f2-ae0f-7babdd39a621/routeoverride-cni/0.log" Jul 13 17:53:38.331842 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:38.331808 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc8ng_e6411f69-741b-44f2-ae0f-7babdd39a621/whereabouts-cni-bincopy/0.log" Jul 13 17:53:38.351044 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:38.351015 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc8ng_e6411f69-741b-44f2-ae0f-7babdd39a621/whereabouts-cni/0.log" Jul 13 17:53:38.676257 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:38.676226 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vpn5b_738b8868-dacb-4206-bb1f-cbc6fdb57ecd/network-metrics-daemon/0.log" Jul 13 17:53:38.692566 ip-10-0-137-245 kubenswrapper[2564]: I0713 17:53:38.692544 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vpn5b_738b8868-dacb-4206-bb1f-cbc6fdb57ecd/kube-rbac-proxy/0.log"