Jul 24 13:32:11.643808 ip-10-0-137-60 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 24 13:32:11.643820 ip-10-0-137-60 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 24 13:32:11.643827 ip-10-0-137-60 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 24 13:32:11.644040 ip-10-0-137-60 systemd[1]: Failed to start Kubernetes Kubelet. Jul 24 13:32:21.893010 ip-10-0-137-60 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 24 13:32:21.893030 ip-10-0-137-60 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot dfc61f16576c4881b737a4458abc833a -- Jul 24 13:34:17.759327 ip-10-0-137-60 systemd[1]: Starting Kubernetes Kubelet... Jul 24 13:34:18.144582 ip-10-0-137-60 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 13:34:18.144582 ip-10-0-137-60 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 24 13:34:18.144582 ip-10-0-137-60 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 13:34:18.144582 ip-10-0-137-60 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 24 13:34:18.144582 ip-10-0-137-60 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 13:34:18.145257 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.144641 2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 24 13:34:18.149403 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149387 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:34:18.149403 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149402 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149406 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149409 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149412 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149415 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149417 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149420 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149424 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149427 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149429 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149432 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149434 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149437 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149440 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149442 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149445 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149448 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149450 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149453 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149455 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:34:18.149469 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149458 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149461 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149463 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149465 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149468 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149471 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149474 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149477 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149480 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149482 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149485 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149487 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149490 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149492 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149495 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149497 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149500 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149503 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149505 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149508 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:34:18.149939 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149510 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149512 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149515 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149519 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149523 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149525 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149528 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149530 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149532 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149535 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149538 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149541 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149543 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149546 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149549 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149552 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149554 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149557 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149559 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:34:18.150462 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149562 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149565 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149567 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149570 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149572 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149574 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149577 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149579 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149582 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149585 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149589 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149592 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149595 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149597 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149600 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149602 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149604 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149607 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149609 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149611 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:34:18.151213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149614 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149617 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149619 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149621 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149625 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149627 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149630 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149634 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149637 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149639 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149642 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149644 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149647 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149649 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149651 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149655 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149658 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149661 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.149664 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150751 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:34:18.151785 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150808 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150813 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150818 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150855 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150859 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150864 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150875 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150879 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150884 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150889 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150892 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150897 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150900 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150906 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150913 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150917 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150921 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150925 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150934 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150939 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:34:18.152334 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150944 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150948 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150953 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150957 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150961 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150965 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150969 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150973 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.150977 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151003 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151013 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151017 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151021 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151025 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151030 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151034 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151068 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151073 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151077 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:34:18.152811 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151081 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151085 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151089 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151093 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151102 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151106 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151111 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151115 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151119 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151123 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151127 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151132 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151136 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151140 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151145 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151149 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151158 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151165 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151171 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:34:18.153289 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151176 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151181 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151185 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151189 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151195 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151200 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151204 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151208 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151212 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151223 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151227 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151260 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151264 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151269 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151272 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151277 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151281 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151285 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151289 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151294 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:34:18.153750 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151297 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151306 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151311 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151315 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151319 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151324 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151328 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151333 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151337 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151342 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151346 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151350 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151354 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151358 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151367 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151371 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151376 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151384 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151389 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.151401 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:34:18.154255 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152042 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152049 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152052 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152055 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152058 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152061 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152064 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152067 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152069 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152072 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152075 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152078 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152080 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152083 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152085 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152088 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152090 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152097 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152100 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152103 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:34:18.154721 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152105 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152108 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152110 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152113 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152115 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152118 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152120 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152123 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152126 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152129 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152131 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152134 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152136 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152139 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152142 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152144 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152147 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152149 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152151 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152155 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:34:18.155215 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152159 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152162 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152165 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152168 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152171 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152173 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152176 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152179 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152181 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152185 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152188 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152190 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152193 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152195 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152198 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152200 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152204 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152208 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152210 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152213 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:34:18.155710 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152215 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152217 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152220 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152222 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152225 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152227 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152229 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152232 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152235 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152238 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152240 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152243 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152245 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152248 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152252 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152255 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152257 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152260 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152262 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152265 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:34:18.156213 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152267 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152271 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152274 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152276 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152278 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152281 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152284 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152287 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152290 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152292 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152295 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152297 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152299 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152302 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152306 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152309 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152312 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152314 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.152317 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:34:18.156686 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152387 2579 flags.go:64] FLAG: --address="0.0.0.0" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152397 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152414 2579 flags.go:64] FLAG: --anonymous-auth="true" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152421 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152427 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152432 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152436 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152442 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152445 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152447 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152451 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152454 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152457 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152460 2579 flags.go:64] FLAG: --cgroup-root="" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152463 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152466 2579 flags.go:64] FLAG: --client-ca-file="" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152475 2579 flags.go:64] FLAG: --cloud-provider="external" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152478 2579 flags.go:64] FLAG: --cluster-dns="[]" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152486 2579 flags.go:64] FLAG: --cluster-domain="" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152489 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152492 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152495 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152498 2579 flags.go:64] FLAG: --container-log-max-files="5" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152501 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 24 13:34:18.157193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152504 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152507 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152510 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152513 2579 flags.go:64] FLAG: --contention-profiling="false" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152516 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152519 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152521 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152524 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152532 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152535 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152538 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152541 2579 flags.go:64] FLAG: --enable-load-reader="false" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152543 2579 flags.go:64] FLAG: --enable-server="true" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152546 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152552 2579 flags.go:64] FLAG: --event-burst="100" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152558 2579 flags.go:64] FLAG: --event-qps="50" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152560 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152563 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152566 2579 flags.go:64] FLAG: --eviction-hard="" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152570 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152572 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152575 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152578 2579 flags.go:64] FLAG: --eviction-soft="" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152581 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152584 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 24 13:34:18.157766 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152592 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152595 2579 flags.go:64] FLAG: --experimental-mounter-path="" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152598 2579 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152601 2579 flags.go:64] FLAG: --fail-swap-on="true" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152603 2579 flags.go:64] FLAG: --feature-gates="" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152607 2579 flags.go:64] FLAG: --file-check-frequency="20s" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152610 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152612 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152615 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152619 2579 flags.go:64] FLAG: --healthz-port="10248" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152622 2579 flags.go:64] FLAG: --help="false" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152624 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-137-60.ec2.internal" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152627 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152630 2579 flags.go:64] FLAG: --http-check-frequency="20s" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152633 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152636 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152640 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152642 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152645 2579 flags.go:64] FLAG: --image-service-endpoint="" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152648 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152650 2579 flags.go:64] FLAG: --kube-api-burst="100" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152653 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152657 2579 flags.go:64] FLAG: --kube-api-qps="50" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152659 2579 flags.go:64] FLAG: --kube-reserved="" Jul 24 13:34:18.158387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152662 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152665 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152667 2579 flags.go:64] FLAG: --kubelet-cgroups="" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152670 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152673 2579 flags.go:64] FLAG: --lock-file="" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152676 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152679 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152682 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152687 2579 flags.go:64] FLAG: --log-json-split-stream="false" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152695 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152698 2579 flags.go:64] FLAG: --log-text-split-stream="false" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152701 2579 flags.go:64] FLAG: --logging-format="text" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152703 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152707 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152710 2579 flags.go:64] FLAG: --manifest-url="" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152712 2579 flags.go:64] FLAG: --manifest-url-header="" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152716 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152719 2579 flags.go:64] FLAG: --max-open-files="1000000" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152723 2579 flags.go:64] FLAG: --max-pods="110" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152726 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152729 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152731 2579 flags.go:64] FLAG: --memory-manager-policy="None" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152734 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152737 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152740 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 24 13:34:18.158958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152742 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152749 2579 flags.go:64] FLAG: --node-status-max-images="50" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152752 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152755 2579 flags.go:64] FLAG: --oom-score-adj="-999" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152757 2579 flags.go:64] FLAG: --pod-cidr="" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152762 2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152767 2579 flags.go:64] FLAG: --pod-manifest-path="" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152770 2579 flags.go:64] FLAG: --pod-max-pids="-1" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152773 2579 flags.go:64] FLAG: --pods-per-core="0" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152776 2579 flags.go:64] FLAG: --port="10250" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152779 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152781 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ae60f3acc6b01caf" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152784 2579 flags.go:64] FLAG: --qos-reserved="" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152787 2579 flags.go:64] FLAG: --read-only-port="10255" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152790 2579 flags.go:64] FLAG: --register-node="true" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152793 2579 flags.go:64] FLAG: --register-with-taints="" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152796 2579 flags.go:64] FLAG: --registry-burst="10" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152805 2579 flags.go:64] FLAG: --registry-qps="5" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152808 2579 flags.go:64] FLAG: --reserved-cpus="" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152811 2579 flags.go:64] FLAG: --reserved-memory="" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152814 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152817 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152820 2579 flags.go:64] FLAG: --rotate-certificates="false" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152823 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152826 2579 flags.go:64] FLAG: --runonce="false" Jul 24 13:34:18.159566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152829 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152832 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152834 2579 flags.go:64] FLAG: --seccomp-default="false" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152837 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152840 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152843 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152846 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152848 2579 flags.go:64] FLAG: --storage-driver-password="root" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152851 2579 flags.go:64] FLAG: --storage-driver-secure="false" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152854 2579 flags.go:64] FLAG: --storage-driver-table="stats" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152856 2579 flags.go:64] FLAG: --storage-driver-user="root" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152859 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152864 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152867 2579 flags.go:64] FLAG: --system-cgroups="" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152870 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152875 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152878 2579 flags.go:64] FLAG: --tls-cert-file="" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152881 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152887 2579 flags.go:64] FLAG: --tls-min-version="" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152890 2579 flags.go:64] FLAG: --tls-private-key-file="" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152893 2579 flags.go:64] FLAG: --topology-manager-policy="none" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152896 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152899 2579 flags.go:64] FLAG: --topology-manager-scope="container" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152902 2579 flags.go:64] FLAG: --v="2" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152906 2579 flags.go:64] FLAG: --version="false" Jul 24 13:34:18.160223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152915 2579 flags.go:64] FLAG: --vmodule="" Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152919 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.152923 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153076 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153081 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153084 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153087 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153090 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153092 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153095 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153097 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153100 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153102 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153104 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153107 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153109 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153112 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153115 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153117 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153122 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:34:18.160841 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153124 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153126 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153129 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153133 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153136 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153139 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153142 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153146 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153150 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153153 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153156 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153159 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153162 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153172 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153175 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153178 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153182 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153185 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153188 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:34:18.161329 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153190 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153193 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153195 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153198 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153200 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153203 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153205 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153208 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153210 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153213 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153216 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153218 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153222 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153224 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153227 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153229 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153232 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153234 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153236 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:34:18.161786 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153239 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153243 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153246 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153248 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153251 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153253 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153255 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153258 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153268 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153271 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153273 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153276 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153278 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153281 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153283 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153286 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153288 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153291 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153293 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153296 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:34:18.162260 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153298 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153301 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153304 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153306 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153309 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153312 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153315 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153317 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153320 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153322 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153325 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153327 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153329 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153333 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153335 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153338 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153340 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153343 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153345 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153348 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:34:18.162730 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153351 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:34:18.163235 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153358 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:34:18.163235 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153361 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:34:18.163235 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.153364 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:34:18.163235 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.154403 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 13:34:18.163235 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.161245 2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 24 13:34:18.163235 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.161261 2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 24 13:34:18.163235 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.161280 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 24 13:34:18.163235 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.161288 2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 24 13:34:18.163235 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161346 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:34:18.163235 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161352 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:34:18.163235 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161355 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:34:18.163235 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161360 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:34:18.163235 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161363 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:34:18.163235 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161366 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161369 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161372 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161375 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161378 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161381 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161384 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161387 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161391 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161395 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161398 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161401 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161404 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161407 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161410 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161413 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161416 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161418 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161421 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:34:18.163583 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161423 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161426 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161429 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161432 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161435 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161439 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161442 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161445 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161448 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161450 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161453 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161455 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161458 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161460 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161463 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161466 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161468 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161471 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161473 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:34:18.164049 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161476 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161478 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161481 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161484 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161486 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161488 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161491 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161493 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161496 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161498 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161500 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161503 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161505 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161508 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161510 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161512 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161515 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161517 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161521 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161524 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:34:18.164537 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161527 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161529 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161532 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161534 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161536 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161539 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161541 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161544 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161546 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161548 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161552 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161555 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161558 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161560 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161563 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161565 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161567 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161570 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161572 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161575 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:34:18.165033 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161577 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:34:18.165506 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161579 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:34:18.165506 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161582 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:34:18.165506 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161584 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:34:18.165506 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161586 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:34:18.165506 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161589 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:34:18.165506 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161591 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:34:18.165506 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161594 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:34:18.165506 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161596 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:34:18.165506 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161598 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:34:18.165506 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161601 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:34:18.165506 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161604 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:34:18.165506 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161608 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:34:18.165506 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161611 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:34:18.165506 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161614 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:34:18.165506 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161616 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:34:18.165983 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.161621 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 13:34:18.165983 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161773 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:34:18.165983 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161780 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:34:18.165983 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161783 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:34:18.165983 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161786 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:34:18.165983 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161789 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:34:18.165983 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161791 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:34:18.165983 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161794 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:34:18.165983 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161797 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:34:18.165983 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161799 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:34:18.165983 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161802 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:34:18.165983 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161804 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:34:18.165983 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161807 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:34:18.165983 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161809 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:34:18.165983 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161812 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161814 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161817 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161819 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161822 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161824 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161826 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161829 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161831 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161834 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161836 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161839 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161842 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161844 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161847 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161850 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161853 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161855 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161857 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161860 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:34:18.166408 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161862 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161864 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161867 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161869 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161871 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161874 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161876 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161878 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161881 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161883 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161885 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161888 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161890 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161892 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161896 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161900 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161902 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161905 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161908 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:34:18.167140 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161910 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161913 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161916 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161918 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161921 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161923 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161926 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161929 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161931 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161934 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161937 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161939 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161941 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161944 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161946 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161948 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161950 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161953 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161955 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161958 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:34:18.167758 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161960 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161962 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161965 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161967 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161970 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161972 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161975 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161978 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161980 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161982 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.161985 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.162000 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.162002 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.162006 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.162009 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.162012 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.162015 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.162018 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.162021 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.162024 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:34:18.168428 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.162026 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:34:18.168906 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.162029 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:34:18.168906 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.162032 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:34:18.168906 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.162034 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:34:18.168906 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.162037 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:34:18.168906 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.162039 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:34:18.168906 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:18.162042 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:34:18.168906 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.162046 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 13:34:18.168906 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.162155 2579 server.go:962] "Client rotation is on, will bootstrap in background" Jul 24 13:34:18.168906 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.164201 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 24 13:34:18.168906 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.165639 2579 server.go:1019] "Starting client certificate rotation" Jul 24 13:34:18.168906 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.165794 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 13:34:18.168906 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.165836 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 13:34:18.168906 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.168432 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 13:34:18.172698 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.172678 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 13:34:18.182325 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.182307 2579 log.go:25] "Validated CRI v1 runtime API" Jul 24 13:34:18.185348 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.185332 2579 log.go:25] "Validated CRI v1 image API" Jul 24 13:34:18.186254 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.186237 2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 24 13:34:18.190630 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.190613 2579 fs.go:135] Filesystem UUIDs: map[3e814ec8-e434-44f2-832a-869f48b3c337:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 df94d98d-ec01-4867-9d8a-e65893bcb913:/dev/nvme0n1p4] Jul 24 13:34:18.190679 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.190630 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 24 13:34:18.196403 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.196295 2579 manager.go:216] Machine: {Timestamp:2026-07-24 13:34:18.194271713 +0000 UTC m=+0.317654102 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099911 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bae4c009e029d14d56afdecf8922f SystemUUID:ec2bae4c-009e-029d-14d5-6afdecf8922f BootID:dfc61f16-576c-4881-b737-a4458abc833a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e8:11:0a:49:11 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e8:11:0a:49:11 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:be:13:cc:e7:79:c2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 24 13:34:18.196403 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.196404 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 24 13:34:18.196507 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.196483 2579 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 24 13:34:18.199120 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.199095 2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 24 13:34:18.199258 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.199124 2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-60.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 24 13:34:18.199776 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.199765 2579 topology_manager.go:138] "Creating topology manager with none policy" Jul 24 13:34:18.199812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.199780 2579 container_manager_linux.go:309] "Creating device plugin manager" Jul 24 13:34:18.199812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.199789 2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 13:34:18.200380 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.200369 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 13:34:18.201233 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.201223 2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 24 13:34:18.203920 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.203909 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 24 13:34:18.204065 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.204054 2579 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 24 13:34:18.204843 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.204831 2579 kubelet.go:486] "Attempting to sync node with API server" Jul 24 13:34:18.204874 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.204849 2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 24 13:34:18.204874 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.204864 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 24 13:34:18.204874 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.204873 2579 kubelet.go:398] "Adding apiserver pod source" Jul 24 13:34:18.204958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.204881 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 24 13:34:18.205250 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.205239 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 13:34:18.205281 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.205257 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 13:34:18.207492 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.207471 2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 24 13:34:18.209196 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.209183 2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 24 13:34:18.209240 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.209213 2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 24 13:34:18.210649 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.210637 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 24 13:34:18.210702 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.210653 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 24 13:34:18.210702 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.210660 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 24 13:34:18.210702 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.210666 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 24 13:34:18.210702 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.210675 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 24 13:34:18.210702 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.210681 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 24 13:34:18.210702 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.210689 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 24 13:34:18.210702 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.210695 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 24 13:34:18.210702 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.210701 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 24 13:34:18.210910 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.210707 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 24 13:34:18.210910 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.210715 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 24 13:34:18.210910 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.210720 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 24 13:34:18.211631 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.211619 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 24 13:34:18.211631 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.211632 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 24 13:34:18.213572 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.213553 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 13:34:18.214847 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.214831 2579 server.go:1268] "Started kubelet" Jul 24 13:34:18.215586 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.215216 2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 24 13:34:18.215586 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.215052 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 24 13:34:18.215768 ip-10-0-137-60 systemd[1]: Started Kubernetes Kubelet. Jul 24 13:34:18.215872 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.215805 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 24 13:34:18.217868 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.217850 2579 server.go:310] "Adding debug handlers to kubelet server" Jul 24 13:34:18.217975 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.217949 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-60.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 24 13:34:18.218110 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.217984 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-60.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 24 13:34:18.218702 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.218648 2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 24 13:34:18.218793 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.218627 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 24 13:34:18.223075 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.223050 2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 24 13:34:18.223596 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.222697 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-60.ec2.internal.18c53d256977a324 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-60.ec2.internal,UID:ip-10-0-137-60.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-60.ec2.internal,},FirstTimestamp:2026-07-24 13:34:18.214802212 +0000 UTC m=+0.338184600,LastTimestamp:2026-07-24 13:34:18.214802212 +0000 UTC m=+0.338184600,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-60.ec2.internal,}" Jul 24 13:34:18.227110 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.227089 2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 24 13:34:18.227110 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.227106 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 24 13:34:18.227436 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.227412 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-60.ec2.internal\" not found" Jul 24 13:34:18.228070 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.228053 2579 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 24 13:34:18.228070 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.228072 2579 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 24 13:34:18.228421 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.228053 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 24 13:34:18.228421 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.228200 2579 reconstruct.go:97] "Volume reconstruction finished" Jul 24 13:34:18.228421 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.228208 2579 reconciler.go:29] "Reconciler: start to sync state" Jul 24 13:34:18.228421 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.228211 2579 factory.go:55] Registering systemd factory Jul 24 13:34:18.228421 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.228229 2579 factory.go:223] Registration of the systemd container factory successfully Jul 24 13:34:18.228637 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.228476 2579 factory.go:153] Registering CRI-O factory Jul 24 13:34:18.228637 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.228487 2579 factory.go:223] Registration of the crio container factory successfully Jul 24 13:34:18.228637 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.228535 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 24 13:34:18.228637 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.228558 2579 factory.go:103] Registering Raw factory Jul 24 13:34:18.228637 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.228567 2579 manager.go:1196] Started watching for new ooms in manager Jul 24 13:34:18.229157 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.229141 2579 manager.go:349] Starting recovery of all containers Jul 24 13:34:18.234692 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.234651 2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-60.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 24 13:34:18.234793 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.234707 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 24 13:34:18.237814 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.237395 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 24 13:34:18.239109 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.239092 2579 manager.go:354] Recovery completed Jul 24 13:34:18.244342 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.244330 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:34:18.246916 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.246893 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:34:18.246976 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.246923 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:34:18.246976 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.246945 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:34:18.247517 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.247503 2579 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 24 13:34:18.247517 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.247516 2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 24 13:34:18.247622 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.247535 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 24 13:34:18.247622 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.247542 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:34:18.248219 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.248203 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:34:18.248290 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.248229 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:34:18.248290 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.248238 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:34:18.249742 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.249662 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-60.ec2.internal.18c53d256b618f8a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-60.ec2.internal,UID:ip-10-0-137-60.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-60.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-60.ec2.internal,},FirstTimestamp:2026-07-24 13:34:18.246909834 +0000 UTC m=+0.370292223,LastTimestamp:2026-07-24 13:34:18.246909834 +0000 UTC m=+0.370292223,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-60.ec2.internal,}" Jul 24 13:34:18.250194 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.250181 2579 policy_none.go:49] "None policy: Start" Jul 24 13:34:18.250254 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.250200 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 24 13:34:18.250254 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.250218 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 24 13:34:18.252186 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.252171 2579 policy_none.go:47] "Start" Jul 24 13:34:18.259495 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.259431 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-60.ec2.internal.18c53d256b61df50 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-60.ec2.internal,UID:ip-10-0-137-60.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-60.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-60.ec2.internal,},FirstTimestamp:2026-07-24 13:34:18.246930256 +0000 UTC m=+0.370312646,LastTimestamp:2026-07-24 13:34:18.246930256 +0000 UTC m=+0.370312646,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-60.ec2.internal,}" Jul 24 13:34:18.272268 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.272198 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-60.ec2.internal.18c53d256b622c2d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-60.ec2.internal,UID:ip-10-0-137-60.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-60.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-60.ec2.internal,},FirstTimestamp:2026-07-24 13:34:18.246949933 +0000 UTC m=+0.370332322,LastTimestamp:2026-07-24 13:34:18.246949933 +0000 UTC m=+0.370332322,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-60.ec2.internal,}" Jul 24 13:34:18.283510 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.283446 2579 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-60.ec2.internal.18c53d256b618f8a\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-60.ec2.internal.18c53d256b618f8a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-60.ec2.internal,UID:ip-10-0-137-60.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-60.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-60.ec2.internal,},FirstTimestamp:2026-07-24 13:34:18.246909834 +0000 UTC m=+0.370292223,LastTimestamp:2026-07-24 13:34:18.248217788 +0000 UTC m=+0.371600177,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-60.ec2.internal,}" Jul 24 13:34:18.292560 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.292546 2579 manager.go:337] "Starting Device Plugin manager" Jul 24 13:34:18.292647 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.292585 2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 24 13:34:18.292647 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.292600 2579 server.go:87] "Starting device plugin registration server" Jul 24 13:34:18.292881 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.292869 2579 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 24 13:34:18.292960 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.292884 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 24 13:34:18.293124 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.293107 2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 24 13:34:18.293217 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.293206 2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 24 13:34:18.293217 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.293217 2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 24 13:34:18.293658 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.293637 2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 24 13:34:18.293778 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.293684 2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-60.ec2.internal\" not found" Jul 24 13:34:18.298945 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.298883 2579 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-60.ec2.internal.18c53d256b61df50\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-60.ec2.internal.18c53d256b61df50 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-60.ec2.internal,UID:ip-10-0-137-60.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-60.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-60.ec2.internal,},FirstTimestamp:2026-07-24 13:34:18.246930256 +0000 UTC m=+0.370312646,LastTimestamp:2026-07-24 13:34:18.24823345 +0000 UTC m=+0.371615839,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-60.ec2.internal,}" Jul 24 13:34:18.324034 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.323942 2579 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-60.ec2.internal.18c53d256b622c2d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-60.ec2.internal.18c53d256b622c2d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-60.ec2.internal,UID:ip-10-0-137-60.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-60.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-60.ec2.internal,},FirstTimestamp:2026-07-24 13:34:18.246949933 +0000 UTC m=+0.370332322,LastTimestamp:2026-07-24 13:34:18.248242335 +0000 UTC m=+0.371624725,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-60.ec2.internal,}" Jul 24 13:34:18.335573 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.335494 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-60.ec2.internal.18c53d256e3ab07c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-60.ec2.internal,UID:ip-10-0-137-60.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-137-60.ec2.internal,},FirstTimestamp:2026-07-24 13:34:18.294694012 +0000 UTC m=+0.418076391,LastTimestamp:2026-07-24 13:34:18.294694012 +0000 UTC m=+0.418076391,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-60.ec2.internal,}" Jul 24 13:34:18.369570 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.369548 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dlzhf" Jul 24 13:34:18.388896 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.388867 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dlzhf" Jul 24 13:34:18.392972 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.392955 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:34:18.394056 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.394037 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:34:18.394149 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.394072 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:34:18.394149 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.394086 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:34:18.394149 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.394118 2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-60.ec2.internal" Jul 24 13:34:18.412695 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.412620 2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-60.ec2.internal" Jul 24 13:34:18.412695 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.412644 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-60.ec2.internal\": node \"ip-10-0-137-60.ec2.internal\" not found" Jul 24 13:34:18.413079 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.413061 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 24 13:34:18.413137 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.413102 2579 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 24 13:34:18.413137 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.413122 2579 kubelet.go:2448] "Starting kubelet main sync loop" Jul 24 13:34:18.413200 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.413158 2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 24 13:34:18.416763 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.416746 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:34:18.514220 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.514186 2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-60.ec2.internal"] Jul 24 13:34:18.514384 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.514283 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:34:18.515241 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.515224 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:34:18.515329 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.515258 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:34:18.515329 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.515269 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:34:18.517432 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.517418 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:34:18.517561 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.517547 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal" Jul 24 13:34:18.517614 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.517584 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:34:18.519202 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.519184 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:34:18.519306 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.519214 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:34:18.519306 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.519224 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:34:18.519306 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.519184 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:34:18.519306 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.519286 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:34:18.519306 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.519298 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:34:18.521540 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.521525 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-60.ec2.internal" Jul 24 13:34:18.521629 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.521552 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:34:18.522324 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.522303 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:34:18.522414 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.522332 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:34:18.522414 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.522346 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:34:18.535560 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.535536 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-60.ec2.internal\" not found" node="ip-10-0-137-60.ec2.internal" Jul 24 13:34:18.539902 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.539885 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-60.ec2.internal\" not found" node="ip-10-0-137-60.ec2.internal" Jul 24 13:34:18.541670 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.541655 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-60.ec2.internal\" not found" Jul 24 13:34:18.629198 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.629170 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d358417e5f98b7abade571df9886aeda-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal\" (UID: \"d358417e5f98b7abade571df9886aeda\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal" Jul 24 13:34:18.629313 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.629201 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d358417e5f98b7abade571df9886aeda-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal\" (UID: \"d358417e5f98b7abade571df9886aeda\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal" Jul 24 13:34:18.629313 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.629226 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2bb58ddfc8f4df1c84ed6456fbda1a1a-config\") pod \"kube-apiserver-proxy-ip-10-0-137-60.ec2.internal\" (UID: \"2bb58ddfc8f4df1c84ed6456fbda1a1a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-60.ec2.internal" Jul 24 13:34:18.642180 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.642158 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-60.ec2.internal\" not found" Jul 24 13:34:18.729807 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.729725 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d358417e5f98b7abade571df9886aeda-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal\" (UID: \"d358417e5f98b7abade571df9886aeda\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal" Jul 24 13:34:18.729807 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.729751 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d358417e5f98b7abade571df9886aeda-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal\" (UID: \"d358417e5f98b7abade571df9886aeda\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal" Jul 24 13:34:18.729807 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.729780 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d358417e5f98b7abade571df9886aeda-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal\" (UID: \"d358417e5f98b7abade571df9886aeda\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal" Jul 24 13:34:18.729807 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.729807 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2bb58ddfc8f4df1c84ed6456fbda1a1a-config\") pod \"kube-apiserver-proxy-ip-10-0-137-60.ec2.internal\" (UID: \"2bb58ddfc8f4df1c84ed6456fbda1a1a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-60.ec2.internal" Jul 24 13:34:18.730054 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.729854 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2bb58ddfc8f4df1c84ed6456fbda1a1a-config\") pod \"kube-apiserver-proxy-ip-10-0-137-60.ec2.internal\" (UID: \"2bb58ddfc8f4df1c84ed6456fbda1a1a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-60.ec2.internal" Jul 24 13:34:18.730054 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.729892 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d358417e5f98b7abade571df9886aeda-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal\" (UID: \"d358417e5f98b7abade571df9886aeda\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal" Jul 24 13:34:18.742837 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.742812 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-60.ec2.internal\" not found" Jul 24 13:34:18.839063 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.839029 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal" Jul 24 13:34:18.842882 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:18.842864 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-60.ec2.internal" Jul 24 13:34:18.843014 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.842956 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-60.ec2.internal\" not found" Jul 24 13:34:18.943293 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:18.943249 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-60.ec2.internal\" not found" Jul 24 13:34:19.043793 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:19.043721 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-60.ec2.internal\" not found" Jul 24 13:34:19.144324 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:19.144296 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-60.ec2.internal\" not found" Jul 24 13:34:19.165498 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:19.165470 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 24 13:34:19.165967 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:19.165687 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 13:34:19.244526 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:19.244492 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-60.ec2.internal\" not found" Jul 24 13:34:19.345141 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:19.345108 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-60.ec2.internal\" not found" Jul 24 13:34:19.394304 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:19.394260 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 13:29:18 +0000 UTC" deadline="2027-12-23 18:55:51.256330569 +0000 UTC" Jul 24 13:34:19.394304 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:19.394296 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12413h21m31.862038456s" Jul 24 13:34:19.427678 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:19.427649 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:34:19.428733 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:19.428713 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:34:19.428863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:19.428751 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:34:19.428863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:19.428766 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:34:19.445185 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:19.445163 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-60.ec2.internal\" not found" Jul 24 13:34:19.445429 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:19.445405 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd358417e5f98b7abade571df9886aeda.slice/crio-931c324a547a778efbc882a01417dbe9e7aa59e116cf396e981a3199e0b13325 WatchSource:0}: Error finding container 931c324a547a778efbc882a01417dbe9e7aa59e116cf396e981a3199e0b13325: Status 404 returned error can't find the container with id 931c324a547a778efbc882a01417dbe9e7aa59e116cf396e981a3199e0b13325 Jul 24 13:34:19.449777 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:19.449757 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:34:19.545598 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:19.545558 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-60.ec2.internal\" not found" Jul 24 13:34:19.553984 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:19.553944 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bb58ddfc8f4df1c84ed6456fbda1a1a.slice/crio-414bfb0f820278dfe1d55ff68ee544b9165ca5725b53191cac3e0e8530653d42 WatchSource:0}: Error finding container 414bfb0f820278dfe1d55ff68ee544b9165ca5725b53191cac3e0e8530653d42: Status 404 returned error can't find the container with id 414bfb0f820278dfe1d55ff68ee544b9165ca5725b53191cac3e0e8530653d42 Jul 24 13:34:19.619954 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:19.619861 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:34:19.624196 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:19.624178 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:34:19.627444 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:19.627426 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal" Jul 24 13:34:19.650121 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:19.650101 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 13:34:19.651188 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:19.651174 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-60.ec2.internal" Jul 24 13:34:19.669234 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:19.669213 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 13:34:19.833095 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:19.833052 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:34:20.206514 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.206478 2579 apiserver.go:52] "Watching apiserver" Jul 24 13:34:20.216504 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.216471 2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 24 13:34:20.216976 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.216941 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-bxddb","kube-system/kube-apiserver-proxy-ip-10-0-137-60.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh","openshift-dns/node-resolver-dkg7g","openshift-image-registry/node-ca-sv57j","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal","openshift-multus/multus-additional-cni-plugins-d6gtn","openshift-network-diagnostics/network-check-target-jkrfc","kube-system/konnectivity-agent-7sc7n","openshift-cluster-node-tuning-operator/tuned-52jwg","openshift-multus/multus-bgf74","openshift-multus/network-metrics-daemon-bsczm","openshift-network-operator/iptables-alerter-4b7cr","openshift-ovn-kubernetes/ovnkube-node-49x5m"] Jul 24 13:34:20.219836 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.219802 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.222353 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.222178 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4b7cr" Jul 24 13:34:20.224797 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.224694 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.228432 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.228412 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 24 13:34:20.229153 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.229028 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dkg7g" Jul 24 13:34:20.231153 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.230905 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 24 13:34:20.231240 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.231185 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 24 13:34:20.231764 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.231348 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 24 13:34:20.231764 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.231678 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 24 13:34:20.231940 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.231908 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 24 13:34:20.233054 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.232133 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 24 13:34:20.233054 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.232633 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zwwtv\"" Jul 24 13:34:20.233054 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.232787 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fb8xr\"" Jul 24 13:34:20.233054 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.232967 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 24 13:34:20.233054 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.232977 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 24 13:34:20.233378 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.233280 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sv57j" Jul 24 13:34:20.238315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.233744 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 24 13:34:20.238315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.233846 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 24 13:34:20.238315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.234017 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 24 13:34:20.238315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.233747 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:34:20.238315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.234121 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 24 13:34:20.238315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.234230 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xpt82\"" Jul 24 13:34:20.238315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.234359 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qh6zf\"" Jul 24 13:34:20.238315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.236727 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 24 13:34:20.238315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.237104 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 24 13:34:20.238315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.237206 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 24 13:34:20.238315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.237264 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5vfnj\"" Jul 24 13:34:20.238315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.237615 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.238315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238115 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-cni-binary-copy\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.238315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238149 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd2kq\" (UniqueName: \"kubernetes.io/projected/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-kube-api-access-zd2kq\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.238315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238207 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6c525745-023d-4cd9-bc32-89f2dec492aa-iptables-alerter-script\") pod \"iptables-alerter-4b7cr\" (UID: \"6c525745-023d-4cd9-bc32-89f2dec492aa\") " pod="openshift-network-operator/iptables-alerter-4b7cr" Jul 24 13:34:20.238315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238241 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcpwb\" (UniqueName: \"kubernetes.io/projected/6c525745-023d-4cd9-bc32-89f2dec492aa-kube-api-access-rcpwb\") pod \"iptables-alerter-4b7cr\" (UID: \"6c525745-023d-4cd9-bc32-89f2dec492aa\") " pod="openshift-network-operator/iptables-alerter-4b7cr" Jul 24 13:34:20.238315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238270 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/96010f19-f46f-4ba7-b907-75d94608d1d0-registration-dir\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.238315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238312 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ea1c6809-b191-43e3-b132-2ceeddc0f058-hosts-file\") pod \"node-resolver-dkg7g\" (UID: \"ea1c6809-b191-43e3-b132-2ceeddc0f058\") " pod="openshift-dns/node-resolver-dkg7g" Jul 24 13:34:20.239285 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238371 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.239285 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238402 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.239285 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238429 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/96010f19-f46f-4ba7-b907-75d94608d1d0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.239285 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238454 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/96010f19-f46f-4ba7-b907-75d94608d1d0-etc-selinux\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.239285 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238480 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/96010f19-f46f-4ba7-b907-75d94608d1d0-sys-fs\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.239285 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238504 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea1c6809-b191-43e3-b132-2ceeddc0f058-tmp-dir\") pod \"node-resolver-dkg7g\" (UID: \"ea1c6809-b191-43e3-b132-2ceeddc0f058\") " pod="openshift-dns/node-resolver-dkg7g" Jul 24 13:34:20.239285 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238529 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxf8v\" (UniqueName: \"kubernetes.io/projected/ea1c6809-b191-43e3-b132-2ceeddc0f058-kube-api-access-nxf8v\") pod \"node-resolver-dkg7g\" (UID: \"ea1c6809-b191-43e3-b132-2ceeddc0f058\") " pod="openshift-dns/node-resolver-dkg7g" Jul 24 13:34:20.239285 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238553 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-cnibin\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.239285 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238579 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c525745-023d-4cd9-bc32-89f2dec492aa-host-slash\") pod \"iptables-alerter-4b7cr\" (UID: \"6c525745-023d-4cd9-bc32-89f2dec492aa\") " pod="openshift-network-operator/iptables-alerter-4b7cr" Jul 24 13:34:20.239285 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238620 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/96010f19-f46f-4ba7-b907-75d94608d1d0-device-dir\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.239285 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238661 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f28cm\" (UniqueName: \"kubernetes.io/projected/96010f19-f46f-4ba7-b907-75d94608d1d0-kube-api-access-f28cm\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.239285 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238698 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b3048916-5eca-431c-84b6-8381da55db53-host\") pod \"node-ca-sv57j\" (UID: \"b3048916-5eca-431c-84b6-8381da55db53\") " pod="openshift-image-registry/node-ca-sv57j" Jul 24 13:34:20.239285 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238742 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b3048916-5eca-431c-84b6-8381da55db53-serviceca\") pod \"node-ca-sv57j\" (UID: \"b3048916-5eca-431c-84b6-8381da55db53\") " pod="openshift-image-registry/node-ca-sv57j" Jul 24 13:34:20.239285 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238788 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cpfw\" (UniqueName: \"kubernetes.io/projected/b3048916-5eca-431c-84b6-8381da55db53-kube-api-access-9cpfw\") pod \"node-ca-sv57j\" (UID: \"b3048916-5eca-431c-84b6-8381da55db53\") " pod="openshift-image-registry/node-ca-sv57j" Jul 24 13:34:20.239285 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238831 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-os-release\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.239285 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238859 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.240142 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238888 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/96010f19-f46f-4ba7-b907-75d94608d1d0-socket-dir\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.240142 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.238926 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-system-cni-dir\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.240243 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.240213 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7sc7n" Jul 24 13:34:20.242812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.241733 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 24 13:34:20.242812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.241826 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 24 13:34:20.242812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.241970 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-27l9r\"" Jul 24 13:34:20.242812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.242143 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 24 13:34:20.242812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.242201 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 24 13:34:20.242812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.242248 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 24 13:34:20.242812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.242452 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 24 13:34:20.243649 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.243529 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 24 13:34:20.243737 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.243679 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 24 13:34:20.244015 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.243935 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:20.245093 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.244372 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5gvh6\"" Jul 24 13:34:20.245093 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:20.245009 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:20.246025 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.245852 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 13:34:20.250235 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.250077 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:20.250705 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:20.250158 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:20.252891 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.252666 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.255599 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.255575 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.256363 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.256347 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cqxlw\"" Jul 24 13:34:20.256573 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.256556 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:34:20.256799 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.256783 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 24 13:34:20.257615 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.257592 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:20.257748 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:20.257726 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:20.259532 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.259500 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lvsn9\"" Jul 24 13:34:20.259773 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.259759 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 24 13:34:20.289098 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.289000 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9xqk9" Jul 24 13:34:20.296092 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.296066 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9xqk9" Jul 24 13:34:20.329596 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.329573 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 24 13:34:20.341041 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.339483 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f28cm\" (UniqueName: \"kubernetes.io/projected/96010f19-f46f-4ba7-b907-75d94608d1d0-kube-api-access-f28cm\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.341041 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.339533 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b3048916-5eca-431c-84b6-8381da55db53-serviceca\") pod \"node-ca-sv57j\" (UID: \"b3048916-5eca-431c-84b6-8381da55db53\") " pod="openshift-image-registry/node-ca-sv57j" Jul 24 13:34:20.341041 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.339567 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-systemd-units\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.341041 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.339596 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-run-ovn-kubernetes\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.341041 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.339625 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-host-run-k8s-cni-cncf-io\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.341041 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.339652 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/96010f19-f46f-4ba7-b907-75d94608d1d0-socket-dir\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.341041 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.339678 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-run-openvswitch\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.341041 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.339703 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-cni-netd\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.341041 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.339733 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-os-release\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.341041 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.339760 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-host-var-lib-cni-multus\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.341041 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.339788 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-cni-binary-copy\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.341041 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.339815 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcpwb\" (UniqueName: \"kubernetes.io/projected/6c525745-023d-4cd9-bc32-89f2dec492aa-kube-api-access-rcpwb\") pod \"iptables-alerter-4b7cr\" (UID: \"6c525745-023d-4cd9-bc32-89f2dec492aa\") " pod="openshift-network-operator/iptables-alerter-4b7cr" Jul 24 13:34:20.341041 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.339852 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/96010f19-f46f-4ba7-b907-75d94608d1d0-registration-dir\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.341041 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.339912 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2ead2935-66f1-4805-9e70-b907d97ccfef-ovn-node-metrics-cert\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.341041 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.339940 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret\") pod \"global-pull-secret-syncer-bxddb\" (UID: \"3fdc7954-a53e-440d-b8fa-889eaf069e9a\") " pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:20.341041 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.339967 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-run\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.341914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340018 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtnb4\" (UniqueName: \"kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4\") pod \"network-check-target-jkrfc\" (UID: \"3c327d99-2456-4a25-9ccf-47f75252aca9\") " pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:20.341914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340068 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea1c6809-b191-43e3-b132-2ceeddc0f058-tmp-dir\") pod \"node-resolver-dkg7g\" (UID: \"ea1c6809-b191-43e3-b132-2ceeddc0f058\") " pod="openshift-dns/node-resolver-dkg7g" Jul 24 13:34:20.341914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340104 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxf8v\" (UniqueName: \"kubernetes.io/projected/ea1c6809-b191-43e3-b132-2ceeddc0f058-kube-api-access-nxf8v\") pod \"node-resolver-dkg7g\" (UID: \"ea1c6809-b191-43e3-b132-2ceeddc0f058\") " pod="openshift-dns/node-resolver-dkg7g" Jul 24 13:34:20.341914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340138 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-etc-openvswitch\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.341914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340164 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnrtr\" (UniqueName: \"kubernetes.io/projected/2ead2935-66f1-4805-9e70-b907d97ccfef-kube-api-access-hnrtr\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.341914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340190 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-tuned\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.341914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340223 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-system-cni-dir\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.341914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340257 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-host-run-netns\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.341914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340283 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cpfw\" (UniqueName: \"kubernetes.io/projected/b3048916-5eca-431c-84b6-8381da55db53-kube-api-access-9cpfw\") pod \"node-ca-sv57j\" (UID: \"b3048916-5eca-431c-84b6-8381da55db53\") " pod="openshift-image-registry/node-ca-sv57j" Jul 24 13:34:20.341914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340309 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-kubelet\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.341914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340336 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3fdc7954-a53e-440d-b8fa-889eaf069e9a-dbus\") pod \"global-pull-secret-syncer-bxddb\" (UID: \"3fdc7954-a53e-440d-b8fa-889eaf069e9a\") " pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:20.341914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340362 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-kubernetes\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.341914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340390 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-sysctl-d\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.341914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340412 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7k9t\" (UniqueName: \"kubernetes.io/projected/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-kube-api-access-v7k9t\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.341914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340429 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.341914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340447 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-system-cni-dir\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.341914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340464 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0c2cbbed-7c36-485b-a62d-bd4abba5d562-agent-certs\") pod \"konnectivity-agent-7sc7n\" (UID: \"0c2cbbed-7c36-485b-a62d-bd4abba5d562\") " pod="kube-system/konnectivity-agent-7sc7n" Jul 24 13:34:20.342812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340492 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-systemd\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.342812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340511 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-sys\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.342812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340537 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-lib-modules\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.342812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340575 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-multus-socket-dir-parent\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.342812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340593 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-host-var-lib-kubelet\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.342812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340610 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/96010f19-f46f-4ba7-b907-75d94608d1d0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.342812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340635 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2ead2935-66f1-4805-9e70-b907d97ccfef-env-overrides\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.342812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340662 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c525745-023d-4cd9-bc32-89f2dec492aa-host-slash\") pod \"iptables-alerter-4b7cr\" (UID: \"6c525745-023d-4cd9-bc32-89f2dec492aa\") " pod="openshift-network-operator/iptables-alerter-4b7cr" Jul 24 13:34:20.342812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340686 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2ead2935-66f1-4805-9e70-b907d97ccfef-ovnkube-config\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.342812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340713 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3fdc7954-a53e-440d-b8fa-889eaf069e9a-kubelet-config\") pod \"global-pull-secret-syncer-bxddb\" (UID: \"3fdc7954-a53e-440d-b8fa-889eaf069e9a\") " pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:20.342812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340747 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-var-lib-kubelet\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.342812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340772 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-host\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.342812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340801 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2ead2935-66f1-4805-9e70-b907d97ccfef-ovnkube-script-lib\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.342812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340828 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs\") pod \"network-metrics-daemon-bsczm\" (UID: \"e2ed2aaf-08f3-4401-ba14-b742f3291961\") " pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:20.342812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340854 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-hostroot\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.342812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340885 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-modprobe-d\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.342812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340910 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-cni-binary-copy\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.343615 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340941 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/96010f19-f46f-4ba7-b907-75d94608d1d0-etc-selinux\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.343615 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.340979 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-run-systemd\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.343615 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341007 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.343615 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341036 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f62xp\" (UniqueName: \"kubernetes.io/projected/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-kube-api-access-f62xp\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.343615 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341092 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/96010f19-f46f-4ba7-b907-75d94608d1d0-device-dir\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.343615 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341123 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b3048916-5eca-431c-84b6-8381da55db53-host\") pod \"node-ca-sv57j\" (UID: \"b3048916-5eca-431c-84b6-8381da55db53\") " pod="openshift-image-registry/node-ca-sv57j" Jul 24 13:34:20.343615 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341149 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-run-ovn\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.343615 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341176 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-log-socket\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.343615 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341205 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-cni-bin\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.343615 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341231 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0c2cbbed-7c36-485b-a62d-bd4abba5d562-konnectivity-ca\") pod \"konnectivity-agent-7sc7n\" (UID: \"0c2cbbed-7c36-485b-a62d-bd4abba5d562\") " pod="kube-system/konnectivity-agent-7sc7n" Jul 24 13:34:20.343615 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341259 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-host-var-lib-cni-bin\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.343615 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341287 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-multus-conf-dir\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.343615 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341396 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-system-cni-dir\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.343615 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341458 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/96010f19-f46f-4ba7-b907-75d94608d1d0-etc-selinux\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.343615 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341598 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/96010f19-f46f-4ba7-b907-75d94608d1d0-socket-dir\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.343615 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341619 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/96010f19-f46f-4ba7-b907-75d94608d1d0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.343615 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341679 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c525745-023d-4cd9-bc32-89f2dec492aa-host-slash\") pod \"iptables-alerter-4b7cr\" (UID: \"6c525745-023d-4cd9-bc32-89f2dec492aa\") " pod="openshift-network-operator/iptables-alerter-4b7cr" Jul 24 13:34:20.344303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341752 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/96010f19-f46f-4ba7-b907-75d94608d1d0-registration-dir\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.344303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341870 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/96010f19-f46f-4ba7-b907-75d94608d1d0-device-dir\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.344303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341911 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b3048916-5eca-431c-84b6-8381da55db53-host\") pod \"node-ca-sv57j\" (UID: \"b3048916-5eca-431c-84b6-8381da55db53\") " pod="openshift-image-registry/node-ca-sv57j" Jul 24 13:34:20.344303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341959 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-os-release\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.344303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.341991 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-tmp\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.344303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342058 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-cnibin\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.344303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342087 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-multus-daemon-config\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.344303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342114 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-etc-kubernetes\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.344303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342232 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b3048916-5eca-431c-84b6-8381da55db53-serviceca\") pod \"node-ca-sv57j\" (UID: \"b3048916-5eca-431c-84b6-8381da55db53\") " pod="openshift-image-registry/node-ca-sv57j" Jul 24 13:34:20.344303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342240 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea1c6809-b191-43e3-b132-2ceeddc0f058-tmp-dir\") pod \"node-resolver-dkg7g\" (UID: \"ea1c6809-b191-43e3-b132-2ceeddc0f058\") " pod="openshift-dns/node-resolver-dkg7g" Jul 24 13:34:20.344303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342252 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd2kq\" (UniqueName: \"kubernetes.io/projected/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-kube-api-access-zd2kq\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.344303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342283 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6c525745-023d-4cd9-bc32-89f2dec492aa-iptables-alerter-script\") pod \"iptables-alerter-4b7cr\" (UID: \"6c525745-023d-4cd9-bc32-89f2dec492aa\") " pod="openshift-network-operator/iptables-alerter-4b7cr" Jul 24 13:34:20.344303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342323 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ea1c6809-b191-43e3-b132-2ceeddc0f058-hosts-file\") pod \"node-resolver-dkg7g\" (UID: \"ea1c6809-b191-43e3-b132-2ceeddc0f058\") " pod="openshift-dns/node-resolver-dkg7g" Jul 24 13:34:20.344303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342326 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-os-release\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.344303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342370 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-var-lib-openvswitch\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.344303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342378 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-cni-binary-copy\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.344303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342452 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-node-log\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.345107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342507 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nbjj\" (UniqueName: \"kubernetes.io/projected/e2ed2aaf-08f3-4401-ba14-b742f3291961-kube-api-access-2nbjj\") pod \"network-metrics-daemon-bsczm\" (UID: \"e2ed2aaf-08f3-4401-ba14-b742f3291961\") " pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:20.345107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342547 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-sysctl-conf\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.345107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342616 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.345107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342624 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-multus-cni-dir\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.345107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342641 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ea1c6809-b191-43e3-b132-2ceeddc0f058-hosts-file\") pod \"node-resolver-dkg7g\" (UID: \"ea1c6809-b191-43e3-b132-2ceeddc0f058\") " pod="openshift-dns/node-resolver-dkg7g" Jul 24 13:34:20.345107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342659 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.345107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342707 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.345107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342754 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/96010f19-f46f-4ba7-b907-75d94608d1d0-sys-fs\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.345107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342810 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.345107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342820 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6c525745-023d-4cd9-bc32-89f2dec492aa-iptables-alerter-script\") pod \"iptables-alerter-4b7cr\" (UID: \"6c525745-023d-4cd9-bc32-89f2dec492aa\") " pod="openshift-network-operator/iptables-alerter-4b7cr" Jul 24 13:34:20.345107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342877 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-slash\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.345107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342900 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/96010f19-f46f-4ba7-b907-75d94608d1d0-sys-fs\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.345107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.342965 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-run-netns\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.345107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.343020 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-sysconfig\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.345107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.343065 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-host-run-multus-certs\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.345107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.343099 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-cnibin\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.345107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.343189 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-cnibin\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.345781 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.343244 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.350445 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.350423 2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 24 13:34:20.355246 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.355222 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcpwb\" (UniqueName: \"kubernetes.io/projected/6c525745-023d-4cd9-bc32-89f2dec492aa-kube-api-access-rcpwb\") pod \"iptables-alerter-4b7cr\" (UID: \"6c525745-023d-4cd9-bc32-89f2dec492aa\") " pod="openshift-network-operator/iptables-alerter-4b7cr" Jul 24 13:34:20.355669 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.355647 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cpfw\" (UniqueName: \"kubernetes.io/projected/b3048916-5eca-431c-84b6-8381da55db53-kube-api-access-9cpfw\") pod \"node-ca-sv57j\" (UID: \"b3048916-5eca-431c-84b6-8381da55db53\") " pod="openshift-image-registry/node-ca-sv57j" Jul 24 13:34:20.357005 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.356981 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f28cm\" (UniqueName: \"kubernetes.io/projected/96010f19-f46f-4ba7-b907-75d94608d1d0-kube-api-access-f28cm\") pod \"aws-ebs-csi-driver-node-gqgxh\" (UID: \"96010f19-f46f-4ba7-b907-75d94608d1d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.357505 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.357480 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxf8v\" (UniqueName: \"kubernetes.io/projected/ea1c6809-b191-43e3-b132-2ceeddc0f058-kube-api-access-nxf8v\") pod \"node-resolver-dkg7g\" (UID: \"ea1c6809-b191-43e3-b132-2ceeddc0f058\") " pod="openshift-dns/node-resolver-dkg7g" Jul 24 13:34:20.358348 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.358328 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd2kq\" (UniqueName: \"kubernetes.io/projected/9832a5d8-4c38-4222-b3cf-18f90bafa6c8-kube-api-access-zd2kq\") pod \"multus-additional-cni-plugins-d6gtn\" (UID: \"9832a5d8-4c38-4222-b3cf-18f90bafa6c8\") " pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.420333 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.419791 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-60.ec2.internal" event={"ID":"2bb58ddfc8f4df1c84ed6456fbda1a1a","Type":"ContainerStarted","Data":"414bfb0f820278dfe1d55ff68ee544b9165ca5725b53191cac3e0e8530653d42"} Jul 24 13:34:20.422163 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.422132 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal" event={"ID":"d358417e5f98b7abade571df9886aeda","Type":"ContainerStarted","Data":"931c324a547a778efbc882a01417dbe9e7aa59e116cf396e981a3199e0b13325"} Jul 24 13:34:20.443829 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.443786 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-var-lib-openvswitch\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.444055 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.443843 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-node-log\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.444055 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.443875 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nbjj\" (UniqueName: \"kubernetes.io/projected/e2ed2aaf-08f3-4401-ba14-b742f3291961-kube-api-access-2nbjj\") pod \"network-metrics-daemon-bsczm\" (UID: \"e2ed2aaf-08f3-4401-ba14-b742f3291961\") " pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:20.444055 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.443907 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-sysctl-conf\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.444055 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.443932 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-multus-cni-dir\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.444055 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.443937 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-node-log\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.444055 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444015 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-multus-cni-dir\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.444055 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444051 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-slash\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.444055 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.443937 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-var-lib-openvswitch\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.444538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444084 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-run-netns\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.444538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444098 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-sysctl-conf\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.444538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444111 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-sysconfig\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.444538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444138 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-host-run-multus-certs\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.444538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444147 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-slash\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.444538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444187 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-systemd-units\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.444538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444198 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-sysconfig\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.444538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444213 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-run-ovn-kubernetes\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.444538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444232 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-host-run-multus-certs\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.444538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444238 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-systemd-units\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.444538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444239 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-host-run-k8s-cni-cncf-io\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.444538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444276 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-host-run-k8s-cni-cncf-io\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.444538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444278 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-run-openvswitch\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.444538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444188 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-run-netns\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.444538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444310 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-run-openvswitch\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.444538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444281 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-run-ovn-kubernetes\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.444538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444322 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-cni-netd\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.444538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444354 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-cni-netd\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.445525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444364 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-os-release\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.445525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444392 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-host-var-lib-cni-multus\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.445525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444425 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-os-release\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.445525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444445 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2ead2935-66f1-4805-9e70-b907d97ccfef-ovn-node-metrics-cert\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.445525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444457 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-host-var-lib-cni-multus\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.445525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444472 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret\") pod \"global-pull-secret-syncer-bxddb\" (UID: \"3fdc7954-a53e-440d-b8fa-889eaf069e9a\") " pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:20.445525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444498 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-run\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.445525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444523 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtnb4\" (UniqueName: \"kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4\") pod \"network-check-target-jkrfc\" (UID: \"3c327d99-2456-4a25-9ccf-47f75252aca9\") " pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:20.445525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444562 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-etc-openvswitch\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.445525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444588 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnrtr\" (UniqueName: \"kubernetes.io/projected/2ead2935-66f1-4805-9e70-b907d97ccfef-kube-api-access-hnrtr\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.445525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444613 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-tuned\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.445525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444648 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-system-cni-dir\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.445525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444674 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-host-run-netns\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.445525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444702 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-kubelet\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.445525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444727 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3fdc7954-a53e-440d-b8fa-889eaf069e9a-dbus\") pod \"global-pull-secret-syncer-bxddb\" (UID: \"3fdc7954-a53e-440d-b8fa-889eaf069e9a\") " pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:20.445525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444751 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-kubernetes\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.445525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444777 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-sysctl-d\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.445525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444800 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7k9t\" (UniqueName: \"kubernetes.io/projected/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-kube-api-access-v7k9t\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.446482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444830 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0c2cbbed-7c36-485b-a62d-bd4abba5d562-agent-certs\") pod \"konnectivity-agent-7sc7n\" (UID: \"0c2cbbed-7c36-485b-a62d-bd4abba5d562\") " pod="kube-system/konnectivity-agent-7sc7n" Jul 24 13:34:20.446482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444844 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-run\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.446482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444854 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-systemd\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.446482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444864 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-system-cni-dir\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.446482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444912 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-kubelet\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.446482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444889 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-kubernetes\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.446482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444930 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-sys\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.446482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444933 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-host-run-netns\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.446482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445070 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3fdc7954-a53e-440d-b8fa-889eaf069e9a-dbus\") pod \"global-pull-secret-syncer-bxddb\" (UID: \"3fdc7954-a53e-440d-b8fa-889eaf069e9a\") " pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:20.446482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445205 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-systemd\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.446482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445244 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-etc-openvswitch\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.446482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445383 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-sysctl-d\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.446482 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:20.444620 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:34:20.446482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.444883 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-sys\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.446482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445489 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-lib-modules\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.446482 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:20.445516 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret podName:3fdc7954-a53e-440d-b8fa-889eaf069e9a nodeName:}" failed. No retries permitted until 2026-07-24 13:34:20.945492455 +0000 UTC m=+3.068874837 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret") pod "global-pull-secret-syncer-bxddb" (UID: "3fdc7954-a53e-440d-b8fa-889eaf069e9a") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:34:20.446482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445549 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-multus-socket-dir-parent\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.446482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445576 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-host-var-lib-kubelet\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.447399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445621 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2ead2935-66f1-4805-9e70-b907d97ccfef-env-overrides\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.447399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445646 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-multus-socket-dir-parent\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.447399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445651 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2ead2935-66f1-4805-9e70-b907d97ccfef-ovnkube-config\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.447399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445687 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3fdc7954-a53e-440d-b8fa-889eaf069e9a-kubelet-config\") pod \"global-pull-secret-syncer-bxddb\" (UID: \"3fdc7954-a53e-440d-b8fa-889eaf069e9a\") " pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:20.447399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445704 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-var-lib-kubelet\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.447399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445721 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-host\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.447399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445794 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-lib-modules\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.447399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445850 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-host-var-lib-kubelet\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.447399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445871 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3fdc7954-a53e-440d-b8fa-889eaf069e9a-kubelet-config\") pod \"global-pull-secret-syncer-bxddb\" (UID: \"3fdc7954-a53e-440d-b8fa-889eaf069e9a\") " pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:20.447399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445905 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-var-lib-kubelet\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.447399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445948 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2ead2935-66f1-4805-9e70-b907d97ccfef-ovnkube-script-lib\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.447399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.445978 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs\") pod \"network-metrics-daemon-bsczm\" (UID: \"e2ed2aaf-08f3-4401-ba14-b742f3291961\") " pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:20.447399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446004 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-hostroot\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.447399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446029 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-modprobe-d\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.447399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446080 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-cni-binary-copy\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.447399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446110 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-run-systemd\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.447399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446139 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.448161 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446170 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f62xp\" (UniqueName: \"kubernetes.io/projected/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-kube-api-access-f62xp\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.448161 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:20.446192 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:34:20.448161 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446199 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-run-ovn\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.448161 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446222 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-log-socket\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.448161 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:20.446256 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs podName:e2ed2aaf-08f3-4401-ba14-b742f3291961 nodeName:}" failed. No retries permitted until 2026-07-24 13:34:20.946234775 +0000 UTC m=+3.069617170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs") pod "network-metrics-daemon-bsczm" (UID: "e2ed2aaf-08f3-4401-ba14-b742f3291961") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:34:20.448161 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446266 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-log-socket\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.448161 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446278 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-cni-bin\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.448161 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446324 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-hostroot\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.448161 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446337 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0c2cbbed-7c36-485b-a62d-bd4abba5d562-konnectivity-ca\") pod \"konnectivity-agent-7sc7n\" (UID: \"0c2cbbed-7c36-485b-a62d-bd4abba5d562\") " pod="kube-system/konnectivity-agent-7sc7n" Jul 24 13:34:20.448161 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446368 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-host-var-lib-cni-bin\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.448161 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446393 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-multus-conf-dir\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.448161 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446417 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-modprobe-d\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.448161 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446424 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-tmp\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.448161 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446449 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-cnibin\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.448161 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446477 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-multus-daemon-config\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.448161 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446492 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2ead2935-66f1-4805-9e70-b907d97ccfef-ovnkube-config\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.448161 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446503 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-etc-kubernetes\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.448161 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446583 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-cni-bin\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.448899 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446624 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.448899 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446668 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-run-systemd\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.448899 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446730 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-cnibin\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.448899 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446747 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-etc-kubernetes\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.448899 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.446969 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-cni-binary-copy\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.448899 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.447135 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-host-var-lib-cni-bin\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.448899 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.447159 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-multus-daemon-config\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.448899 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.447200 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2ead2935-66f1-4805-9e70-b907d97ccfef-run-ovn\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.448899 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.447258 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-host\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.448899 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.447258 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-multus-conf-dir\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.448899 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.448657 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0c2cbbed-7c36-485b-a62d-bd4abba5d562-agent-certs\") pod \"konnectivity-agent-7sc7n\" (UID: \"0c2cbbed-7c36-485b-a62d-bd4abba5d562\") " pod="kube-system/konnectivity-agent-7sc7n" Jul 24 13:34:20.449465 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.449334 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-etc-tuned\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.451073 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.451029 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2ead2935-66f1-4805-9e70-b907d97ccfef-env-overrides\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.451215 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.451200 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-tmp\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.452421 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.452395 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0c2cbbed-7c36-485b-a62d-bd4abba5d562-konnectivity-ca\") pod \"konnectivity-agent-7sc7n\" (UID: \"0c2cbbed-7c36-485b-a62d-bd4abba5d562\") " pod="kube-system/konnectivity-agent-7sc7n" Jul 24 13:34:20.454308 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.454283 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2ead2935-66f1-4805-9e70-b907d97ccfef-ovn-node-metrics-cert\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.454454 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.454428 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2ead2935-66f1-4805-9e70-b907d97ccfef-ovnkube-script-lib\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.458490 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.458435 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nbjj\" (UniqueName: \"kubernetes.io/projected/e2ed2aaf-08f3-4401-ba14-b742f3291961-kube-api-access-2nbjj\") pod \"network-metrics-daemon-bsczm\" (UID: \"e2ed2aaf-08f3-4401-ba14-b742f3291961\") " pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:20.463949 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:20.463923 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:34:20.464093 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:20.463957 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:34:20.464093 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:20.463973 2579 projected.go:196] Error preparing data for projected volume kube-api-access-mtnb4 for pod openshift-network-diagnostics/network-check-target-jkrfc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:34:20.464093 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:20.464062 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4 podName:3c327d99-2456-4a25-9ccf-47f75252aca9 nodeName:}" failed. No retries permitted until 2026-07-24 13:34:20.964042484 +0000 UTC m=+3.087424890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mtnb4" (UniqueName: "kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4") pod "network-check-target-jkrfc" (UID: "3c327d99-2456-4a25-9ccf-47f75252aca9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:34:20.467195 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.467174 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f62xp\" (UniqueName: \"kubernetes.io/projected/1dbad8af-8632-4e52-9f7f-55f6b0e75be1-kube-api-access-f62xp\") pod \"tuned-52jwg\" (UID: \"1dbad8af-8632-4e52-9f7f-55f6b0e75be1\") " pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.467823 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.467803 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7k9t\" (UniqueName: \"kubernetes.io/projected/1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610-kube-api-access-v7k9t\") pod \"multus-bgf74\" (UID: \"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610\") " pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.469917 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.469891 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnrtr\" (UniqueName: \"kubernetes.io/projected/2ead2935-66f1-4805-9e70-b907d97ccfef-kube-api-access-hnrtr\") pod \"ovnkube-node-49x5m\" (UID: \"2ead2935-66f1-4805-9e70-b907d97ccfef\") " pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.536472 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.536438 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d6gtn" Jul 24 13:34:20.544880 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.544855 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:34:20.547402 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.547373 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4b7cr" Jul 24 13:34:20.562371 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.562339 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" Jul 24 13:34:20.568537 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.568512 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dkg7g" Jul 24 13:34:20.577408 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.577389 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sv57j" Jul 24 13:34:20.588360 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.588341 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:20.599381 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.599362 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7sc7n" Jul 24 13:34:20.616152 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.616128 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-52jwg" Jul 24 13:34:20.624138 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.624116 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bgf74" Jul 24 13:34:20.951581 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.951501 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret\") pod \"global-pull-secret-syncer-bxddb\" (UID: \"3fdc7954-a53e-440d-b8fa-889eaf069e9a\") " pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:20.951581 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:20.951571 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs\") pod \"network-metrics-daemon-bsczm\" (UID: \"e2ed2aaf-08f3-4401-ba14-b742f3291961\") " pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:20.951782 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:20.951676 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:34:20.951782 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:20.951695 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:34:20.951782 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:20.951741 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret podName:3fdc7954-a53e-440d-b8fa-889eaf069e9a nodeName:}" failed. No retries permitted until 2026-07-24 13:34:21.951725005 +0000 UTC m=+4.075107398 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret") pod "global-pull-secret-syncer-bxddb" (UID: "3fdc7954-a53e-440d-b8fa-889eaf069e9a") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:34:20.951782 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:20.951766 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs podName:e2ed2aaf-08f3-4401-ba14-b742f3291961 nodeName:}" failed. No retries permitted until 2026-07-24 13:34:21.951747374 +0000 UTC m=+4.075129770 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs") pod "network-metrics-daemon-bsczm" (UID: "e2ed2aaf-08f3-4401-ba14-b742f3291961") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:34:21.052617 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:21.052576 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtnb4\" (UniqueName: \"kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4\") pod \"network-check-target-jkrfc\" (UID: \"3c327d99-2456-4a25-9ccf-47f75252aca9\") " pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:21.052805 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:21.052748 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:34:21.052805 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:21.052769 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:34:21.052805 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:21.052781 2579 projected.go:196] Error preparing data for projected volume kube-api-access-mtnb4 for pod openshift-network-diagnostics/network-check-target-jkrfc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:34:21.052995 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:21.052847 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4 podName:3c327d99-2456-4a25-9ccf-47f75252aca9 nodeName:}" failed. No retries permitted until 2026-07-24 13:34:22.052825261 +0000 UTC m=+4.176207679 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mtnb4" (UniqueName: "kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4") pod "network-check-target-jkrfc" (UID: "3c327d99-2456-4a25-9ccf-47f75252aca9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:34:21.299857 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:21.299727 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 13:29:20 +0000 UTC" deadline="2028-04-18 03:41:53.581361492 +0000 UTC" Jul 24 13:34:21.299857 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:21.299801 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15206h7m32.281566738s" Jul 24 13:34:21.413985 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:21.413954 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:21.414182 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:21.414133 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:21.620207 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:21.620138 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c525745_023d_4cd9_bc32_89f2dec492aa.slice/crio-b919b53c800d4eb7a723de0639fe2847f7d463b3180a1032abde2149fadfb242 WatchSource:0}: Error finding container b919b53c800d4eb7a723de0639fe2847f7d463b3180a1032abde2149fadfb242: Status 404 returned error can't find the container with id b919b53c800d4eb7a723de0639fe2847f7d463b3180a1032abde2149fadfb242 Jul 24 13:34:21.625262 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:21.625105 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea1c6809_b191_43e3_b132_2ceeddc0f058.slice/crio-77d879999db719e3ef41b915693c10ce6e20c2374d29274ddadebb315098a99a WatchSource:0}: Error finding container 77d879999db719e3ef41b915693c10ce6e20c2374d29274ddadebb315098a99a: Status 404 returned error can't find the container with id 77d879999db719e3ef41b915693c10ce6e20c2374d29274ddadebb315098a99a Jul 24 13:34:21.627958 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:21.627918 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d6fa8a3_ac5a_48c7_ae04_0f2f25e3d610.slice/crio-31bc1c14178543d397c90d70201f1ceb5e5bce189aec390b8dd59ad9a9ddcdca WatchSource:0}: Error finding container 31bc1c14178543d397c90d70201f1ceb5e5bce189aec390b8dd59ad9a9ddcdca: Status 404 returned error can't find the container with id 31bc1c14178543d397c90d70201f1ceb5e5bce189aec390b8dd59ad9a9ddcdca Jul 24 13:34:21.638614 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:21.638583 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96010f19_f46f_4ba7_b907_75d94608d1d0.slice/crio-21ccaeb652de97ddf259ecbe58016c525165f15a84a7103e4b8505d59de45dbb WatchSource:0}: Error finding container 21ccaeb652de97ddf259ecbe58016c525165f15a84a7103e4b8505d59de45dbb: Status 404 returned error can't find the container with id 21ccaeb652de97ddf259ecbe58016c525165f15a84a7103e4b8505d59de45dbb Jul 24 13:34:21.639232 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:21.639209 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3048916_5eca_431c_84b6_8381da55db53.slice/crio-4dae83ce3da4f95aaa7f106b54e52012f4934cdc7838b24b98d505c55ba45658 WatchSource:0}: Error finding container 4dae83ce3da4f95aaa7f106b54e52012f4934cdc7838b24b98d505c55ba45658: Status 404 returned error can't find the container with id 4dae83ce3da4f95aaa7f106b54e52012f4934cdc7838b24b98d505c55ba45658 Jul 24 13:34:21.640975 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:21.640950 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ead2935_66f1_4805_9e70_b907d97ccfef.slice/crio-335f44ce2b7b69568b68ebeb297941c632031cf88325f2677a5f24ba61b304fb WatchSource:0}: Error finding container 335f44ce2b7b69568b68ebeb297941c632031cf88325f2677a5f24ba61b304fb: Status 404 returned error can't find the container with id 335f44ce2b7b69568b68ebeb297941c632031cf88325f2677a5f24ba61b304fb Jul 24 13:34:21.642640 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:34:21.642427 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9832a5d8_4c38_4222_b3cf_18f90bafa6c8.slice/crio-6d29b674ecacb2c2158392f7d641d454f8f2edf3bb91e21dc4537fb236854dfb WatchSource:0}: Error finding container 6d29b674ecacb2c2158392f7d641d454f8f2edf3bb91e21dc4537fb236854dfb: Status 404 returned error can't find the container with id 6d29b674ecacb2c2158392f7d641d454f8f2edf3bb91e21dc4537fb236854dfb Jul 24 13:34:21.960898 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:21.960851 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs\") pod \"network-metrics-daemon-bsczm\" (UID: \"e2ed2aaf-08f3-4401-ba14-b742f3291961\") " pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:21.961086 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:21.960909 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret\") pod \"global-pull-secret-syncer-bxddb\" (UID: \"3fdc7954-a53e-440d-b8fa-889eaf069e9a\") " pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:21.961086 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:21.961040 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:34:21.961086 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:21.961041 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:34:21.961188 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:21.961098 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret podName:3fdc7954-a53e-440d-b8fa-889eaf069e9a nodeName:}" failed. No retries permitted until 2026-07-24 13:34:23.961083942 +0000 UTC m=+6.084466323 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret") pod "global-pull-secret-syncer-bxddb" (UID: "3fdc7954-a53e-440d-b8fa-889eaf069e9a") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:34:21.961188 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:21.961115 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs podName:e2ed2aaf-08f3-4401-ba14-b742f3291961 nodeName:}" failed. No retries permitted until 2026-07-24 13:34:23.961107614 +0000 UTC m=+6.084489993 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs") pod "network-metrics-daemon-bsczm" (UID: "e2ed2aaf-08f3-4401-ba14-b742f3291961") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:34:22.061307 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:22.061267 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtnb4\" (UniqueName: \"kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4\") pod \"network-check-target-jkrfc\" (UID: \"3c327d99-2456-4a25-9ccf-47f75252aca9\") " pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:22.061501 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:22.061468 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:34:22.061501 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:22.061497 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:34:22.061617 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:22.061512 2579 projected.go:196] Error preparing data for projected volume kube-api-access-mtnb4 for pod openshift-network-diagnostics/network-check-target-jkrfc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:34:22.061617 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:22.061585 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4 podName:3c327d99-2456-4a25-9ccf-47f75252aca9 nodeName:}" failed. No retries permitted until 2026-07-24 13:34:24.061563378 +0000 UTC m=+6.184945759 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mtnb4" (UniqueName: "kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4") pod "network-check-target-jkrfc" (UID: "3c327d99-2456-4a25-9ccf-47f75252aca9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:34:22.300626 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:22.300502 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 13:29:20 +0000 UTC" deadline="2028-01-28 17:08:38.457957174 +0000 UTC" Jul 24 13:34:22.300626 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:22.300571 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13275h34m16.157389099s" Jul 24 13:34:22.421501 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:22.421284 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:22.421676 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:22.421534 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:22.422243 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:22.422127 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:22.422366 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:22.422238 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:22.439538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:22.438054 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-60.ec2.internal" event={"ID":"2bb58ddfc8f4df1c84ed6456fbda1a1a","Type":"ContainerStarted","Data":"6011d18e532e8bc48c7f7b8fd20507ba8508f1dbfbf8e825569b6d02bcfd9dfd"} Jul 24 13:34:22.445190 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:22.445162 2579 generic.go:356] "Generic (PLEG): container finished" podID="d358417e5f98b7abade571df9886aeda" containerID="3d997f055c1ce5321d6585a9620c944797808a35ab668c356dbb877502dc313d" exitCode=0 Jul 24 13:34:22.445307 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:22.445242 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal" event={"ID":"d358417e5f98b7abade571df9886aeda","Type":"ContainerDied","Data":"3d997f055c1ce5321d6585a9620c944797808a35ab668c356dbb877502dc313d"} Jul 24 13:34:22.450657 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:22.450603 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6gtn" event={"ID":"9832a5d8-4c38-4222-b3cf-18f90bafa6c8","Type":"ContainerStarted","Data":"6d29b674ecacb2c2158392f7d641d454f8f2edf3bb91e21dc4537fb236854dfb"} Jul 24 13:34:22.460190 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:22.459718 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" event={"ID":"2ead2935-66f1-4805-9e70-b907d97ccfef","Type":"ContainerStarted","Data":"335f44ce2b7b69568b68ebeb297941c632031cf88325f2677a5f24ba61b304fb"} Jul 24 13:34:22.462770 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:22.462609 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-60.ec2.internal" podStartSLOduration=3.462590176 podStartE2EDuration="3.462590176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:34:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:34:22.461681548 +0000 UTC m=+4.585063950" watchObservedRunningTime="2026-07-24 13:34:22.462590176 +0000 UTC m=+4.585972578" Jul 24 13:34:22.486906 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:22.486868 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bgf74" event={"ID":"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610","Type":"ContainerStarted","Data":"31bc1c14178543d397c90d70201f1ceb5e5bce189aec390b8dd59ad9a9ddcdca"} Jul 24 13:34:22.496676 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:22.495306 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dkg7g" event={"ID":"ea1c6809-b191-43e3-b132-2ceeddc0f058","Type":"ContainerStarted","Data":"77d879999db719e3ef41b915693c10ce6e20c2374d29274ddadebb315098a99a"} Jul 24 13:34:22.514328 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:22.514021 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sv57j" event={"ID":"b3048916-5eca-431c-84b6-8381da55db53","Type":"ContainerStarted","Data":"4dae83ce3da4f95aaa7f106b54e52012f4934cdc7838b24b98d505c55ba45658"} Jul 24 13:34:22.543227 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:22.543186 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7sc7n" event={"ID":"0c2cbbed-7c36-485b-a62d-bd4abba5d562","Type":"ContainerStarted","Data":"3b62e6bc5decd95c9cfd562a8b91d420279710d92247600ca3de1d36081e517b"} Jul 24 13:34:22.553745 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:22.553667 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" event={"ID":"96010f19-f46f-4ba7-b907-75d94608d1d0","Type":"ContainerStarted","Data":"21ccaeb652de97ddf259ecbe58016c525165f15a84a7103e4b8505d59de45dbb"} Jul 24 13:34:22.564314 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:22.564273 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-52jwg" event={"ID":"1dbad8af-8632-4e52-9f7f-55f6b0e75be1","Type":"ContainerStarted","Data":"4ecafe8a1023c5a3aa1cb94a4c2a6526535733a4f2be5dd5566491e833aa045e"} Jul 24 13:34:22.585610 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:22.585573 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4b7cr" event={"ID":"6c525745-023d-4cd9-bc32-89f2dec492aa","Type":"ContainerStarted","Data":"b919b53c800d4eb7a723de0639fe2847f7d463b3180a1032abde2149fadfb242"} Jul 24 13:34:23.413890 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:23.413854 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:23.414369 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:23.414028 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:23.609372 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:23.609335 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal" event={"ID":"d358417e5f98b7abade571df9886aeda","Type":"ContainerStarted","Data":"7ab6fad6e83b96e4fb8697ee1c79721b9b8b6089f5af15b14a60566253df7787"} Jul 24 13:34:23.977278 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:23.976418 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs\") pod \"network-metrics-daemon-bsczm\" (UID: \"e2ed2aaf-08f3-4401-ba14-b742f3291961\") " pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:23.977278 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:23.976527 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret\") pod \"global-pull-secret-syncer-bxddb\" (UID: \"3fdc7954-a53e-440d-b8fa-889eaf069e9a\") " pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:23.977278 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:23.976665 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:34:23.977278 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:23.976740 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret podName:3fdc7954-a53e-440d-b8fa-889eaf069e9a nodeName:}" failed. No retries permitted until 2026-07-24 13:34:27.976721421 +0000 UTC m=+10.100103814 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret") pod "global-pull-secret-syncer-bxddb" (UID: "3fdc7954-a53e-440d-b8fa-889eaf069e9a") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:34:23.977278 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:23.977180 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:34:23.977278 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:23.977237 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs podName:e2ed2aaf-08f3-4401-ba14-b742f3291961 nodeName:}" failed. No retries permitted until 2026-07-24 13:34:27.977220198 +0000 UTC m=+10.100602578 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs") pod "network-metrics-daemon-bsczm" (UID: "e2ed2aaf-08f3-4401-ba14-b742f3291961") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:34:24.077144 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:24.077060 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtnb4\" (UniqueName: \"kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4\") pod \"network-check-target-jkrfc\" (UID: \"3c327d99-2456-4a25-9ccf-47f75252aca9\") " pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:24.077311 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:24.077263 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:34:24.077311 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:24.077284 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:34:24.077311 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:24.077296 2579 projected.go:196] Error preparing data for projected volume kube-api-access-mtnb4 for pod openshift-network-diagnostics/network-check-target-jkrfc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:34:24.077475 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:24.077368 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4 podName:3c327d99-2456-4a25-9ccf-47f75252aca9 nodeName:}" failed. No retries permitted until 2026-07-24 13:34:28.077348062 +0000 UTC m=+10.200730444 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mtnb4" (UniqueName: "kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4") pod "network-check-target-jkrfc" (UID: "3c327d99-2456-4a25-9ccf-47f75252aca9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:34:24.416047 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:24.415951 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:24.416489 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:24.416101 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:24.416489 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:24.416448 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:24.416594 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:24.416536 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:25.414057 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:25.413527 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:25.414057 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:25.413674 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:26.415137 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:26.415106 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:26.415465 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:26.415242 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:26.415943 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:26.415725 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:26.415943 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:26.415828 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:27.413824 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:27.413787 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:27.414006 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:27.413935 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:28.008381 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:28.008344 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret\") pod \"global-pull-secret-syncer-bxddb\" (UID: \"3fdc7954-a53e-440d-b8fa-889eaf069e9a\") " pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:28.008814 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:28.008421 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs\") pod \"network-metrics-daemon-bsczm\" (UID: \"e2ed2aaf-08f3-4401-ba14-b742f3291961\") " pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:28.008814 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:28.008508 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:34:28.008814 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:28.008542 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:34:28.008814 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:28.008571 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret podName:3fdc7954-a53e-440d-b8fa-889eaf069e9a nodeName:}" failed. No retries permitted until 2026-07-24 13:34:36.008555283 +0000 UTC m=+18.131937662 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret") pod "global-pull-secret-syncer-bxddb" (UID: "3fdc7954-a53e-440d-b8fa-889eaf069e9a") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:34:28.008814 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:28.008593 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs podName:e2ed2aaf-08f3-4401-ba14-b742f3291961 nodeName:}" failed. No retries permitted until 2026-07-24 13:34:36.008577381 +0000 UTC m=+18.131959762 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs") pod "network-metrics-daemon-bsczm" (UID: "e2ed2aaf-08f3-4401-ba14-b742f3291961") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:34:28.109284 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:28.109249 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtnb4\" (UniqueName: \"kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4\") pod \"network-check-target-jkrfc\" (UID: \"3c327d99-2456-4a25-9ccf-47f75252aca9\") " pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:28.109454 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:28.109433 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:34:28.109454 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:28.109455 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:34:28.109568 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:28.109464 2579 projected.go:196] Error preparing data for projected volume kube-api-access-mtnb4 for pod openshift-network-diagnostics/network-check-target-jkrfc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:34:28.109568 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:28.109515 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4 podName:3c327d99-2456-4a25-9ccf-47f75252aca9 nodeName:}" failed. No retries permitted until 2026-07-24 13:34:36.109495372 +0000 UTC m=+18.232877752 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mtnb4" (UniqueName: "kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4") pod "network-check-target-jkrfc" (UID: "3c327d99-2456-4a25-9ccf-47f75252aca9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:34:28.415130 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:28.415047 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:28.415273 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:28.415170 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:28.416302 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:28.416278 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:28.416436 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:28.416374 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:29.414659 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:29.414167 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:29.414659 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:29.414312 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:30.414264 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:30.414232 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:30.414445 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:30.414376 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:30.414445 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:30.414425 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:30.414581 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:30.414556 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:31.413873 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:31.413360 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:31.413873 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:31.413503 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:32.413819 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:32.413747 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:32.414009 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:32.413893 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:32.414009 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:32.413973 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:32.414364 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:32.414083 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:33.413473 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:33.413438 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:33.413668 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:33.413583 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:34.414015 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:34.413502 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:34.414015 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:34.413633 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:34.414015 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:34.413699 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:34.414015 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:34.413790 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:35.413503 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:35.413461 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:35.413720 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:35.413608 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:36.071734 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:36.071695 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs\") pod \"network-metrics-daemon-bsczm\" (UID: \"e2ed2aaf-08f3-4401-ba14-b742f3291961\") " pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:36.072122 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:36.071754 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret\") pod \"global-pull-secret-syncer-bxddb\" (UID: \"3fdc7954-a53e-440d-b8fa-889eaf069e9a\") " pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:36.072122 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:36.071835 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:34:36.072122 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:36.071841 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:34:36.072122 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:36.071884 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret podName:3fdc7954-a53e-440d-b8fa-889eaf069e9a nodeName:}" failed. No retries permitted until 2026-07-24 13:34:52.071870751 +0000 UTC m=+34.195253128 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret") pod "global-pull-secret-syncer-bxddb" (UID: "3fdc7954-a53e-440d-b8fa-889eaf069e9a") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:34:36.072122 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:36.071931 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs podName:e2ed2aaf-08f3-4401-ba14-b742f3291961 nodeName:}" failed. No retries permitted until 2026-07-24 13:34:52.071918975 +0000 UTC m=+34.195301350 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs") pod "network-metrics-daemon-bsczm" (UID: "e2ed2aaf-08f3-4401-ba14-b742f3291961") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:34:36.173175 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:36.173136 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtnb4\" (UniqueName: \"kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4\") pod \"network-check-target-jkrfc\" (UID: \"3c327d99-2456-4a25-9ccf-47f75252aca9\") " pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:36.173314 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:36.173270 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:34:36.173314 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:36.173284 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:34:36.173314 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:36.173292 2579 projected.go:196] Error preparing data for projected volume kube-api-access-mtnb4 for pod openshift-network-diagnostics/network-check-target-jkrfc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:34:36.173414 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:36.173346 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4 podName:3c327d99-2456-4a25-9ccf-47f75252aca9 nodeName:}" failed. No retries permitted until 2026-07-24 13:34:52.173333534 +0000 UTC m=+34.296715926 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mtnb4" (UniqueName: "kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4") pod "network-check-target-jkrfc" (UID: "3c327d99-2456-4a25-9ccf-47f75252aca9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:34:36.413719 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:36.413622 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:36.413883 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:36.413631 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:36.413883 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:36.413774 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:36.413883 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:36.413836 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:37.413811 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:37.413776 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:37.414319 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:37.413916 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:38.414326 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:38.414292 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:38.414674 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:38.414395 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:38.414852 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:38.414793 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:38.414967 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:38.414898 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:39.414253 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:39.414221 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:39.414455 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:39.414331 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:40.413642 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.413339 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:40.413821 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.413582 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:40.413821 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:40.413759 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:40.413939 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:40.413840 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:40.645772 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.645746 2579 generic.go:356] "Generic (PLEG): container finished" podID="9832a5d8-4c38-4222-b3cf-18f90bafa6c8" containerID="311f8aab38dce3077aebe9551d3d879522d1ee06b97f6d527b64322f5e38c282" exitCode=0 Jul 24 13:34:40.646429 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.645832 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6gtn" event={"ID":"9832a5d8-4c38-4222-b3cf-18f90bafa6c8","Type":"ContainerDied","Data":"311f8aab38dce3077aebe9551d3d879522d1ee06b97f6d527b64322f5e38c282"} Jul 24 13:34:40.648621 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.648597 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" event={"ID":"2ead2935-66f1-4805-9e70-b907d97ccfef","Type":"ContainerStarted","Data":"79f0b721c6c87d99aad81bb321ba55b7c20bbbe3f67127efa22be1fe1bebf978"} Jul 24 13:34:40.648710 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.648671 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" event={"ID":"2ead2935-66f1-4805-9e70-b907d97ccfef","Type":"ContainerStarted","Data":"8328273f5dd15f64e0ab267e760ffc7d95faf2a541e617d48f0922a035d3ee6f"} Jul 24 13:34:40.648811 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.648719 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" event={"ID":"2ead2935-66f1-4805-9e70-b907d97ccfef","Type":"ContainerStarted","Data":"12397f0a292aa21136020144ced3fe8b817eecd80570a42df1bf97d240b307b9"} Jul 24 13:34:40.648811 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.648775 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" event={"ID":"2ead2935-66f1-4805-9e70-b907d97ccfef","Type":"ContainerStarted","Data":"ded0792544de44b246fc41296309dfc74f0c100317749c757061fed15819eeb2"} Jul 24 13:34:40.650096 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.650072 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bgf74" event={"ID":"1d6fa8a3-ac5a-48c7-ae04-0f2f25e3d610","Type":"ContainerStarted","Data":"6caa85931dad62cbf53d7f185066b2ad41db13d673084be2d65d2743020428db"} Jul 24 13:34:40.651260 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.651231 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dkg7g" event={"ID":"ea1c6809-b191-43e3-b132-2ceeddc0f058","Type":"ContainerStarted","Data":"9b399b137af65d529cff915ff2ad3975d9e1a0d4f82498604986acd40cdd3b2b"} Jul 24 13:34:40.652657 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.652634 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sv57j" event={"ID":"b3048916-5eca-431c-84b6-8381da55db53","Type":"ContainerStarted","Data":"a193b5a3bd4421ba4b8985198a0a7d2bea603acf41fcae900fa1e0778a20d0e7"} Jul 24 13:34:40.653904 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.653886 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7sc7n" event={"ID":"0c2cbbed-7c36-485b-a62d-bd4abba5d562","Type":"ContainerStarted","Data":"1d424bba85dc9115d81a86797f9f21a0bf385582c6a5dbe3c0a1b867f87e42f7"} Jul 24 13:34:40.655111 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.655092 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" event={"ID":"96010f19-f46f-4ba7-b907-75d94608d1d0","Type":"ContainerStarted","Data":"76e3289d7c4c9110029019a2db1a5eb0db6ee753fa4b2d9e4e53ad684ebaee33"} Jul 24 13:34:40.656690 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.656346 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-52jwg" event={"ID":"1dbad8af-8632-4e52-9f7f-55f6b0e75be1","Type":"ContainerStarted","Data":"81ca9e8e32e79afb32fce4cb6ca9e1ec790f362822ad469c724e53c97ebcd8c8"} Jul 24 13:34:40.672977 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.672939 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-60.ec2.internal" podStartSLOduration=21.672927469 podStartE2EDuration="21.672927469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:34:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:34:23.630033234 +0000 UTC m=+5.753415663" watchObservedRunningTime="2026-07-24 13:34:40.672927469 +0000 UTC m=+22.796310017" Jul 24 13:34:40.683144 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.683124 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7sc7n" Jul 24 13:34:40.683549 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.683533 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7sc7n" Jul 24 13:34:40.712246 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.712206 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bgf74" podStartSLOduration=4.144868707 podStartE2EDuration="22.712191895s" totalImagesPullingTime="18.567323188s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:34:40.712016004 +0000 UTC m=+22.835398403" watchObservedRunningTime="2026-07-24 13:34:40.712191895 +0000 UTC m=+22.835574300" Jul 24 13:34:40.712665 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.712628 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sv57j" podStartSLOduration=4.227893498 podStartE2EDuration="22.712620119s" totalImagesPullingTime="18.484726621s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:34:40.69203976 +0000 UTC m=+22.815422158" watchObservedRunningTime="2026-07-24 13:34:40.712620119 +0000 UTC m=+22.836002517" Jul 24 13:34:40.734090 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.734053 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dkg7g" podStartSLOduration=4.205701443 podStartE2EDuration="22.734039311s" totalImagesPullingTime="18.528337868s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:34:40.733829104 +0000 UTC m=+22.857211501" watchObservedRunningTime="2026-07-24 13:34:40.734039311 +0000 UTC m=+22.857421709" Jul 24 13:34:40.749640 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.749604 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7sc7n" podStartSLOduration=4.227719939 podStartE2EDuration="22.749590814s" totalImagesPullingTime="18.521870875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:34:40.749339599 +0000 UTC m=+22.872721997" watchObservedRunningTime="2026-07-24 13:34:40.749590814 +0000 UTC m=+22.872973249" Jul 24 13:34:40.770003 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.769949 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-52jwg" podStartSLOduration=4.224210596 podStartE2EDuration="22.769934398s" totalImagesPullingTime="18.545723802s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:34:40.769488819 +0000 UTC m=+22.892871219" watchObservedRunningTime="2026-07-24 13:34:40.769934398 +0000 UTC m=+22.893316796" Jul 24 13:34:40.866887 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.866853 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:40.961174 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:40.961095 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:41.413627 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:41.413597 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:41.413800 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:41.413723 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:41.665340 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:41.665308 2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 24 13:34:41.671914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:41.671832 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" event={"ID":"2ead2935-66f1-4805-9e70-b907d97ccfef","Type":"ContainerStarted","Data":"dc4f273529850f0759a91e2893f14789bff293aec5aff25c74c03f31ad00b054"} Jul 24 13:34:41.672035 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:41.671952 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" event={"ID":"2ead2935-66f1-4805-9e70-b907d97ccfef","Type":"ContainerStarted","Data":"ab7f304057c446b52777f145ddaab958cd739c102d3991015e0f7a9fdccffee4"} Jul 24 13:34:41.674166 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:41.674140 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" event={"ID":"96010f19-f46f-4ba7-b907-75d94608d1d0","Type":"ContainerStarted","Data":"b206803332dd180de1d33cdb038a0ae1fe88f4619f3a9e5433bf1c09d136aa2c"} Jul 24 13:34:41.677078 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:41.677057 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4b7cr" event={"ID":"6c525745-023d-4cd9-bc32-89f2dec492aa","Type":"ContainerStarted","Data":"24736bbf8bec0341759dcaa65461b724e65b153f627aee4ec8de7e4f75cf5496"} Jul 24 13:34:41.677397 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:41.677262 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7sc7n" Jul 24 13:34:41.677743 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:41.677723 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7sc7n" Jul 24 13:34:41.723364 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:41.723306 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4b7cr" podStartSLOduration=5.1950329140000004 podStartE2EDuration="23.723288439s" totalImagesPullingTime="18.528255525s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:34:41.700892279 +0000 UTC m=+23.824274679" watchObservedRunningTime="2026-07-24 13:34:41.723288439 +0000 UTC m=+23.846670837" Jul 24 13:34:41.792052 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:41.791974 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:41.893784 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:41.893706 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:42.001823 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:42.001782 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:42.112062 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:42.112024 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:42.214447 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:42.214358 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:42.312790 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:42.312682 2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T13:34:41.665337155Z","UUID":"f394f871-f178-4b6c-9ea0-72775dad1eda","Handler":null,"Name":"","Endpoint":""} Jul 24 13:34:42.314607 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:42.314586 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 24 13:34:42.314726 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:42.314620 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 24 13:34:42.323539 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:42.323405 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:42.413621 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:42.413587 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:42.413797 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:42.413626 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:42.413797 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:42.413733 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:42.413912 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:42.413878 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:42.493527 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:42.493452 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:42.624763 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:42.624718 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:42.856785 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:42.856752 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:43.026477 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:43.026433 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:43.414404 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:43.414332 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:43.414626 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:43.414454 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:43.595818 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:43.595780 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:43.683548 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:43.683465 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" event={"ID":"96010f19-f46f-4ba7-b907-75d94608d1d0","Type":"ContainerStarted","Data":"10c160506caed699183a101527902fe0a2b6901bb5867f156af862a4455a44e8"} Jul 24 13:34:43.687491 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:43.687456 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" event={"ID":"2ead2935-66f1-4805-9e70-b907d97ccfef","Type":"ContainerStarted","Data":"40408faf9a902397843f3905049c92d18fafce32a0364374c231a1a926c09de9"} Jul 24 13:34:43.736400 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:43.736365 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:43.855795 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:43.855737 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:43.929545 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:43.929502 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:44.360545 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:44.360380 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dkg7g_ea1c6809-b191-43e3-b132-2ceeddc0f058/dns-node-resolver/0.log" Jul 24 13:34:44.413851 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:44.413819 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:44.414015 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:44.413943 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:44.414015 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:44.414006 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:44.414159 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:44.414120 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:44.523590 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:44.523551 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:44.623322 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:44.623217 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:44.725047 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:44.725005 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:45.054771 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:45.054723 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:45.170253 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:45.170217 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:45.276176 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:45.276074 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:45.370079 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:45.370048 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:45.414040 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:45.414009 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:45.414215 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:45.414129 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:45.694491 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:45.694463 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6gtn" event={"ID":"9832a5d8-4c38-4222-b3cf-18f90bafa6c8","Type":"ContainerStarted","Data":"26a71f9ff77fba55cbc8fc17349f348c3d02f434af900fdf5eb5695b165e73be"} Jul 24 13:34:45.698444 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:45.698412 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" event={"ID":"2ead2935-66f1-4805-9e70-b907d97ccfef","Type":"ContainerStarted","Data":"7e48421e5673f2ad09fecb8e482cccf07a070f3f22efac4c00d249db6aa1e891"} Jul 24 13:34:45.698704 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:45.698680 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:45.714495 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:45.714463 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:45.728132 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:45.728089 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqgxh" podStartSLOduration=6.604757017 podStartE2EDuration="27.728072112s" totalImagesPullingTime="21.123315095s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:34:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:34:43.75629105 +0000 UTC m=+25.879673445" watchObservedRunningTime="2026-07-24 13:34:45.728072112 +0000 UTC m=+27.851454563" Jul 24 13:34:45.767355 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:45.767310 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" podStartSLOduration=9.218761907 podStartE2EDuration="27.767293511s" totalImagesPullingTime="18.548531604s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:34:45.765167122 +0000 UTC m=+27.888549514" watchObservedRunningTime="2026-07-24 13:34:45.767293511 +0000 UTC m=+27.890675908" Jul 24 13:34:45.775162 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:45.775145 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sv57j_b3048916-5eca-431c-84b6-8381da55db53/node-ca/0.log" Jul 24 13:34:46.163508 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:46.163344 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:46.264423 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:46.264388 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:46.414022 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:46.413938 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:46.414166 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:46.414049 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:46.414166 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:46.414139 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:46.414239 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:46.414219 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:46.599094 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:46.599055 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:46.702754 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:46.702663 2579 generic.go:356] "Generic (PLEG): container finished" podID="9832a5d8-4c38-4222-b3cf-18f90bafa6c8" containerID="26a71f9ff77fba55cbc8fc17349f348c3d02f434af900fdf5eb5695b165e73be" exitCode=0 Jul 24 13:34:46.702754 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:46.702742 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6gtn" event={"ID":"9832a5d8-4c38-4222-b3cf-18f90bafa6c8","Type":"ContainerDied","Data":"26a71f9ff77fba55cbc8fc17349f348c3d02f434af900fdf5eb5695b165e73be"} Jul 24 13:34:46.703428 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:46.703411 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:46.703544 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:46.703488 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:46.723531 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:46.723503 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:34:47.282718 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:47.282684 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:47.413513 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:47.413483 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:47.413668 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:47.413600 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:47.425783 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:47.425751 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:47.550643 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:47.550564 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:47.605810 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:47.605780 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bxddb"] Jul 24 13:34:47.605953 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:47.605878 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:47.606012 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:47.605958 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:47.610365 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:47.610336 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jkrfc"] Jul 24 13:34:47.610507 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:47.610445 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:47.610597 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:47.610561 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:47.610977 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:47.610959 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bsczm"] Jul 24 13:34:47.657209 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:47.657174 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:47.705103 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:47.705059 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:47.705301 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:47.705236 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:47.855089 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:47.854976 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:48.184469 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:48.184384 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:48.710077 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:48.709973 2579 generic.go:356] "Generic (PLEG): container finished" podID="9832a5d8-4c38-4222-b3cf-18f90bafa6c8" containerID="513904e8c3e5981fe4540aab46288df52702fa6150971c343389fbe3a9385287" exitCode=0 Jul 24 13:34:48.710077 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:48.710027 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6gtn" event={"ID":"9832a5d8-4c38-4222-b3cf-18f90bafa6c8","Type":"ContainerDied","Data":"513904e8c3e5981fe4540aab46288df52702fa6150971c343389fbe3a9385287"} Jul 24 13:34:48.788162 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:48.788125 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:49.207692 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:49.207498 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:49.413447 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:49.413365 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:49.413447 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:49.413389 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:49.413447 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:49.413371 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:49.413674 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:49.413484 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:49.413674 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:49.413539 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:49.413674 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:49.413621 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:49.434904 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:49.434872 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:49.683608 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:49.683576 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:49.714671 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:49.714642 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6gtn" event={"ID":"9832a5d8-4c38-4222-b3cf-18f90bafa6c8","Type":"ContainerStarted","Data":"7c684ca27ede7a5dc780ded99f2b3e6ada4e7069a9eaa195af7a49022bafd4e2"} Jul 24 13:34:49.813452 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:49.813423 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:50.391401 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:50.391369 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:50.719045 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:50.718945 2579 generic.go:356] "Generic (PLEG): container finished" podID="9832a5d8-4c38-4222-b3cf-18f90bafa6c8" containerID="7c684ca27ede7a5dc780ded99f2b3e6ada4e7069a9eaa195af7a49022bafd4e2" exitCode=0 Jul 24 13:34:50.719045 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:50.719029 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6gtn" event={"ID":"9832a5d8-4c38-4222-b3cf-18f90bafa6c8","Type":"ContainerDied","Data":"7c684ca27ede7a5dc780ded99f2b3e6ada4e7069a9eaa195af7a49022bafd4e2"} Jul 24 13:34:50.969542 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:50.969458 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:51.069434 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:51.069404 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:51.171450 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:51.171417 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:51.413617 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:51.413536 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:51.413779 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:51.413658 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:51.413779 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:51.413653 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:51.413779 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:51.413705 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:51.413926 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:51.413787 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:51.413926 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:51.413876 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:51.591026 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:51.590969 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:51.782084 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:51.781986 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:51.889059 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:51.889022 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:52.095684 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:52.095606 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs\") pod \"network-metrics-daemon-bsczm\" (UID: \"e2ed2aaf-08f3-4401-ba14-b742f3291961\") " pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:52.095684 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:52.095660 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret\") pod \"global-pull-secret-syncer-bxddb\" (UID: \"3fdc7954-a53e-440d-b8fa-889eaf069e9a\") " pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:52.095931 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:52.095755 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:34:52.095931 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:52.095775 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:34:52.095931 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:52.095815 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret podName:3fdc7954-a53e-440d-b8fa-889eaf069e9a nodeName:}" failed. No retries permitted until 2026-07-24 13:35:24.09579648 +0000 UTC m=+66.219178858 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret") pod "global-pull-secret-syncer-bxddb" (UID: "3fdc7954-a53e-440d-b8fa-889eaf069e9a") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:34:52.095931 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:52.095839 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs podName:e2ed2aaf-08f3-4401-ba14-b742f3291961 nodeName:}" failed. No retries permitted until 2026-07-24 13:35:24.095822102 +0000 UTC m=+66.219204498 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs") pod "network-metrics-daemon-bsczm" (UID: "e2ed2aaf-08f3-4401-ba14-b742f3291961") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:34:52.196148 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:52.196111 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtnb4\" (UniqueName: \"kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4\") pod \"network-check-target-jkrfc\" (UID: \"3c327d99-2456-4a25-9ccf-47f75252aca9\") " pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:52.196313 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:52.196259 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:34:52.196313 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:52.196277 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:34:52.196313 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:52.196286 2579 projected.go:196] Error preparing data for projected volume kube-api-access-mtnb4 for pod openshift-network-diagnostics/network-check-target-jkrfc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:34:52.196424 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:52.196344 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4 podName:3c327d99-2456-4a25-9ccf-47f75252aca9 nodeName:}" failed. No retries permitted until 2026-07-24 13:35:24.196327166 +0000 UTC m=+66.319709563 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mtnb4" (UniqueName: "kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4") pod "network-check-target-jkrfc" (UID: "3c327d99-2456-4a25-9ccf-47f75252aca9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:34:52.373118 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:52.373038 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:52.800207 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:52.800121 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:53.229303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:53.229160 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:53.357466 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:53.357427 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:53.414009 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:53.413958 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:53.414191 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:53.414023 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:53.414191 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:53.414023 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:53.414191 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:53.414122 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:53.414355 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:53.414206 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:53.414355 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:53.414305 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:53.489516 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:53.489434 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:53.994635 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:53.994593 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:54.380398 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:54.380312 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:55.193486 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:55.193455 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:55.293807 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:55.293782 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:55.414088 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:55.413981 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:55.414243 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:55.413981 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:55.414243 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:55.414110 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:55.414243 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:55.413981 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:55.414243 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:55.414205 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:55.414386 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:55.414259 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:55.734378 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:55.734298 2579 generic.go:356] "Generic (PLEG): container finished" podID="9832a5d8-4c38-4222-b3cf-18f90bafa6c8" containerID="f0b34633effd62ff26e44a77f2caed69cb34ef476221b7fad95c662b6f7b00ab" exitCode=0 Jul 24 13:34:55.734378 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:55.734354 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6gtn" event={"ID":"9832a5d8-4c38-4222-b3cf-18f90bafa6c8","Type":"ContainerDied","Data":"f0b34633effd62ff26e44a77f2caed69cb34ef476221b7fad95c662b6f7b00ab"} Jul 24 13:34:55.857664 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:55.857634 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:56.034066 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:56.033963 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:56.210776 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:56.210742 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:56.611822 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:56.611794 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:56.739461 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:56.739434 2579 generic.go:356] "Generic (PLEG): container finished" podID="9832a5d8-4c38-4222-b3cf-18f90bafa6c8" containerID="5d76d2712558acd5ae75d96944bf93c8dc5c019ae0ad8f3607194e1b38c4be8c" exitCode=0 Jul 24 13:34:56.739590 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:56.739479 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6gtn" event={"ID":"9832a5d8-4c38-4222-b3cf-18f90bafa6c8","Type":"ContainerDied","Data":"5d76d2712558acd5ae75d96944bf93c8dc5c019ae0ad8f3607194e1b38c4be8c"} Jul 24 13:34:57.379434 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:57.379396 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:57.414407 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:57.414330 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:57.414567 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:57.414431 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:57.414567 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:57.414529 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:34:57.414567 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:57.414434 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:57.414567 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:57.414439 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:57.414737 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:57.414624 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:57.524041 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:57.523983 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:57.657103 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:57.657070 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:57.744749 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:57.744663 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6gtn" event={"ID":"9832a5d8-4c38-4222-b3cf-18f90bafa6c8","Type":"ContainerStarted","Data":"155e093d4ce351b9586409763a7826f8a871d01a3a465004fa023df5f86c21d0"} Jul 24 13:34:57.795947 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:57.795910 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:58.000168 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:58.000075 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:59.115980 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:59.115942 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:59.375741 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:59.375656 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:34:59.413537 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:59.413507 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:34:59.413680 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:59.413603 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:34:59.413680 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:34:59.413611 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:34:59.413750 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:59.413607 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:34:59.413750 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:59.413700 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:34:59.413812 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:34:59.413751 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:35:00.235973 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:00.235942 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:00.337939 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:00.337896 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:01.413903 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:01.413866 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:35:01.414278 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:01.413868 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:35:01.414278 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:01.413966 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:35:01.414278 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:01.414063 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:35:01.414278 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:01.413869 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:35:01.414278 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:01.414153 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:35:03.413303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:03.413269 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:35:03.413303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:03.413301 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:35:03.413303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:03.413313 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:35:03.413808 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:03.413384 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:35:03.413808 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:03.413448 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:35:03.413808 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:03.413525 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:35:04.264217 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:04.264186 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:04.362267 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:04.362237 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:04.564145 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:04.564110 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:04.813846 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:04.813782 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:04.885845 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:04.885763 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:05.222518 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:05.222427 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:05.327682 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:05.327639 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:05.413947 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:05.413918 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:35:05.414161 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:05.414021 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:35:05.414161 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:05.414030 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:35:05.414161 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:05.414122 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:35:05.414284 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:05.414156 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:35:05.414284 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:05.414250 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:35:05.417482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:05.417452 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:05.932748 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:05.932718 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:06.153306 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:06.153270 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:06.452691 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:06.452655 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:06.527532 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:06.527504 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:07.124662 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:07.124632 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:07.250233 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:07.250205 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:07.413875 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:07.413790 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:35:07.414053 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:07.413790 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:35:07.414053 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:07.413906 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bsczm" podUID="e2ed2aaf-08f3-4401-ba14-b742f3291961" Jul 24 13:35:07.414053 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:07.413790 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:35:07.414053 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:07.414010 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jkrfc" podUID="3c327d99-2456-4a25-9ccf-47f75252aca9" Jul 24 13:35:07.414186 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:07.414074 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bxddb" podUID="3fdc7954-a53e-440d-b8fa-889eaf069e9a" Jul 24 13:35:07.422416 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:07.422387 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:07.582420 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:07.582388 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:07.723282 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:07.723250 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:08.149461 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.149435 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-60.ec2.internal" event="NodeReady" Jul 24 13:35:08.149826 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.149567 2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 24 13:35:08.199639 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.199586 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-d6gtn" podStartSLOduration=19.884976438 podStartE2EDuration="50.199568731s" totalImagesPullingTime="30.314592293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:34:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:34:57.782500856 +0000 UTC m=+39.905883255" watchObservedRunningTime="2026-07-24 13:35:08.199568731 +0000 UTC m=+50.322951129" Jul 24 13:35:08.200174 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.200159 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c46b45557-8b5kp"] Jul 24 13:35:08.214868 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.214838 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.219105 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.219079 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 24 13:35:08.219244 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.219079 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 24 13:35:08.219394 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.219382 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 24 13:35:08.220043 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.220024 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hklft\"" Jul 24 13:35:08.227795 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.227260 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dj997"] Jul 24 13:35:08.228188 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.228167 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 24 13:35:08.247894 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.247866 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c46b45557-8b5kp"] Jul 24 13:35:08.247894 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.247901 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zz9l9"] Jul 24 13:35:08.248069 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.248019 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dj997" Jul 24 13:35:08.251883 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.251784 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 24 13:35:08.251883 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.251852 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wnnsn\"" Jul 24 13:35:08.252089 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.251893 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 24 13:35:08.252330 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.252136 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 24 13:35:08.252395 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.252366 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 24 13:35:08.271767 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.271740 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zz9l9"] Jul 24 13:35:08.271870 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.271776 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dj997"] Jul 24 13:35:08.271870 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.271862 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zz9l9" Jul 24 13:35:08.276857 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.276817 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 24 13:35:08.277013 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.276977 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 24 13:35:08.277204 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.277189 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-h9qpm\"" Jul 24 13:35:08.277285 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.277257 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 24 13:35:08.313365 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.313337 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ck595"] Jul 24 13:35:08.316807 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.316785 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1e0fc9a-b19b-4a76-aead-44c891fc755a-installation-pull-secrets\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.316876 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.316818 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1e0fc9a-b19b-4a76-aead-44c891fc755a-registry-certificates\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.316876 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.316834 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1e0fc9a-b19b-4a76-aead-44c891fc755a-trusted-ca\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.316876 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.316850 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdqxt\" (UniqueName: \"kubernetes.io/projected/c1e0fc9a-b19b-4a76-aead-44c891fc755a-kube-api-access-hdqxt\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.317024 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.316900 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1e0fc9a-b19b-4a76-aead-44c891fc755a-ca-trust-extracted\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.317024 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.316951 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1e0fc9a-b19b-4a76-aead-44c891fc755a-bound-sa-token\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.317135 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.317039 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1e0fc9a-b19b-4a76-aead-44c891fc755a-image-registry-private-configuration\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.317135 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.317076 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1e0fc9a-b19b-4a76-aead-44c891fc755a-registry-tls\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.333093 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.333071 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ck595"] Jul 24 13:35:08.333213 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.333163 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ck595" Jul 24 13:35:08.337238 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.337214 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 24 13:35:08.337366 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.337300 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kzzx2\"" Jul 24 13:35:08.337522 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.337509 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 24 13:35:08.417639 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.417560 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3579589-b0f5-4d12-998d-892f6e682086-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dj997\" (UID: \"c3579589-b0f5-4d12-998d-892f6e682086\") " pod="openshift-insights/insights-runtime-extractor-dj997" Jul 24 13:35:08.417639 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.417599 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1e0fc9a-b19b-4a76-aead-44c891fc755a-bound-sa-token\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.417639 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.417619 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3579589-b0f5-4d12-998d-892f6e682086-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dj997\" (UID: \"c3579589-b0f5-4d12-998d-892f6e682086\") " pod="openshift-insights/insights-runtime-extractor-dj997" Jul 24 13:35:08.417908 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.417642 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1e0fc9a-b19b-4a76-aead-44c891fc755a-image-registry-private-configuration\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.417908 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.417661 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1e0fc9a-b19b-4a76-aead-44c891fc755a-registry-tls\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.417908 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.417704 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/005b211d-95c2-4052-972d-61ac571c313b-cert\") pod \"ingress-canary-zz9l9\" (UID: \"005b211d-95c2-4052-972d-61ac571c313b\") " pod="openshift-ingress-canary/ingress-canary-zz9l9" Jul 24 13:35:08.417908 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.417821 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1e0fc9a-b19b-4a76-aead-44c891fc755a-installation-pull-secrets\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.417908 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.417860 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsjxh\" (UniqueName: \"kubernetes.io/projected/005b211d-95c2-4052-972d-61ac571c313b-kube-api-access-jsjxh\") pod \"ingress-canary-zz9l9\" (UID: \"005b211d-95c2-4052-972d-61ac571c313b\") " pod="openshift-ingress-canary/ingress-canary-zz9l9" Jul 24 13:35:08.417908 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.417887 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rwm5\" (UniqueName: \"kubernetes.io/projected/c3579589-b0f5-4d12-998d-892f6e682086-kube-api-access-8rwm5\") pod \"insights-runtime-extractor-dj997\" (UID: \"c3579589-b0f5-4d12-998d-892f6e682086\") " pod="openshift-insights/insights-runtime-extractor-dj997" Jul 24 13:35:08.418174 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.417920 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1e0fc9a-b19b-4a76-aead-44c891fc755a-registry-certificates\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.418174 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.417977 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1e0fc9a-b19b-4a76-aead-44c891fc755a-trusted-ca\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.418174 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.418024 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdqxt\" (UniqueName: \"kubernetes.io/projected/c1e0fc9a-b19b-4a76-aead-44c891fc755a-kube-api-access-hdqxt\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.418174 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.418053 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1e0fc9a-b19b-4a76-aead-44c891fc755a-ca-trust-extracted\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.418451 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.418430 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c3579589-b0f5-4d12-998d-892f6e682086-crio-socket\") pod \"insights-runtime-extractor-dj997\" (UID: \"c3579589-b0f5-4d12-998d-892f6e682086\") " pod="openshift-insights/insights-runtime-extractor-dj997" Jul 24 13:35:08.418519 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.418486 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3579589-b0f5-4d12-998d-892f6e682086-data-volume\") pod \"insights-runtime-extractor-dj997\" (UID: \"c3579589-b0f5-4d12-998d-892f6e682086\") " pod="openshift-insights/insights-runtime-extractor-dj997" Jul 24 13:35:08.418860 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.418836 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1e0fc9a-b19b-4a76-aead-44c891fc755a-ca-trust-extracted\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.419480 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.419448 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1e0fc9a-b19b-4a76-aead-44c891fc755a-trusted-ca\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.423351 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.422641 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1e0fc9a-b19b-4a76-aead-44c891fc755a-image-registry-private-configuration\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.423351 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.422670 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1e0fc9a-b19b-4a76-aead-44c891fc755a-registry-tls\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.423351 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.422681 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1e0fc9a-b19b-4a76-aead-44c891fc755a-installation-pull-secrets\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.428714 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.428681 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1e0fc9a-b19b-4a76-aead-44c891fc755a-registry-certificates\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.435650 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.435627 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1e0fc9a-b19b-4a76-aead-44c891fc755a-bound-sa-token\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.435853 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.435834 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdqxt\" (UniqueName: \"kubernetes.io/projected/c1e0fc9a-b19b-4a76-aead-44c891fc755a-kube-api-access-hdqxt\") pod \"image-registry-6c46b45557-8b5kp\" (UID: \"c1e0fc9a-b19b-4a76-aead-44c891fc755a\") " pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.519930 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.519887 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsjxh\" (UniqueName: \"kubernetes.io/projected/005b211d-95c2-4052-972d-61ac571c313b-kube-api-access-jsjxh\") pod \"ingress-canary-zz9l9\" (UID: \"005b211d-95c2-4052-972d-61ac571c313b\") " pod="openshift-ingress-canary/ingress-canary-zz9l9" Jul 24 13:35:08.519930 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.519939 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rwm5\" (UniqueName: \"kubernetes.io/projected/c3579589-b0f5-4d12-998d-892f6e682086-kube-api-access-8rwm5\") pod \"insights-runtime-extractor-dj997\" (UID: \"c3579589-b0f5-4d12-998d-892f6e682086\") " pod="openshift-insights/insights-runtime-extractor-dj997" Jul 24 13:35:08.520224 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.519979 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjgf4\" (UniqueName: \"kubernetes.io/projected/db90cec8-5114-4f7f-8966-79193d3cc584-kube-api-access-fjgf4\") pod \"dns-default-ck595\" (UID: \"db90cec8-5114-4f7f-8966-79193d3cc584\") " pod="openshift-dns/dns-default-ck595" Jul 24 13:35:08.520224 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.520045 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c3579589-b0f5-4d12-998d-892f6e682086-crio-socket\") pod \"insights-runtime-extractor-dj997\" (UID: \"c3579589-b0f5-4d12-998d-892f6e682086\") " pod="openshift-insights/insights-runtime-extractor-dj997" Jul 24 13:35:08.520224 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.520067 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3579589-b0f5-4d12-998d-892f6e682086-data-volume\") pod \"insights-runtime-extractor-dj997\" (UID: \"c3579589-b0f5-4d12-998d-892f6e682086\") " pod="openshift-insights/insights-runtime-extractor-dj997" Jul 24 13:35:08.520224 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.520087 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db90cec8-5114-4f7f-8966-79193d3cc584-metrics-tls\") pod \"dns-default-ck595\" (UID: \"db90cec8-5114-4f7f-8966-79193d3cc584\") " pod="openshift-dns/dns-default-ck595" Jul 24 13:35:08.520224 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.520111 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3579589-b0f5-4d12-998d-892f6e682086-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dj997\" (UID: \"c3579589-b0f5-4d12-998d-892f6e682086\") " pod="openshift-insights/insights-runtime-extractor-dj997" Jul 24 13:35:08.520224 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.520150 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3579589-b0f5-4d12-998d-892f6e682086-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dj997\" (UID: \"c3579589-b0f5-4d12-998d-892f6e682086\") " pod="openshift-insights/insights-runtime-extractor-dj997" Jul 24 13:35:08.520525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.520232 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/db90cec8-5114-4f7f-8966-79193d3cc584-config-volume\") pod \"dns-default-ck595\" (UID: \"db90cec8-5114-4f7f-8966-79193d3cc584\") " pod="openshift-dns/dns-default-ck595" Jul 24 13:35:08.520525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.520266 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c3579589-b0f5-4d12-998d-892f6e682086-crio-socket\") pod \"insights-runtime-extractor-dj997\" (UID: \"c3579589-b0f5-4d12-998d-892f6e682086\") " pod="openshift-insights/insights-runtime-extractor-dj997" Jul 24 13:35:08.520525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.520273 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/005b211d-95c2-4052-972d-61ac571c313b-cert\") pod \"ingress-canary-zz9l9\" (UID: \"005b211d-95c2-4052-972d-61ac571c313b\") " pod="openshift-ingress-canary/ingress-canary-zz9l9" Jul 24 13:35:08.520525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.520355 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db90cec8-5114-4f7f-8966-79193d3cc584-tmp-dir\") pod \"dns-default-ck595\" (UID: \"db90cec8-5114-4f7f-8966-79193d3cc584\") " pod="openshift-dns/dns-default-ck595" Jul 24 13:35:08.520525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.520474 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3579589-b0f5-4d12-998d-892f6e682086-data-volume\") pod \"insights-runtime-extractor-dj997\" (UID: \"c3579589-b0f5-4d12-998d-892f6e682086\") " pod="openshift-insights/insights-runtime-extractor-dj997" Jul 24 13:35:08.520807 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.520787 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3579589-b0f5-4d12-998d-892f6e682086-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dj997\" (UID: \"c3579589-b0f5-4d12-998d-892f6e682086\") " pod="openshift-insights/insights-runtime-extractor-dj997" Jul 24 13:35:08.521742 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.521717 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:08.522934 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.522906 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3579589-b0f5-4d12-998d-892f6e682086-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dj997\" (UID: \"c3579589-b0f5-4d12-998d-892f6e682086\") " pod="openshift-insights/insights-runtime-extractor-dj997" Jul 24 13:35:08.523114 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.523095 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/005b211d-95c2-4052-972d-61ac571c313b-cert\") pod \"ingress-canary-zz9l9\" (UID: \"005b211d-95c2-4052-972d-61ac571c313b\") " pod="openshift-ingress-canary/ingress-canary-zz9l9" Jul 24 13:35:08.526880 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.526838 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:08.530093 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.530075 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rwm5\" (UniqueName: \"kubernetes.io/projected/c3579589-b0f5-4d12-998d-892f6e682086-kube-api-access-8rwm5\") pod \"insights-runtime-extractor-dj997\" (UID: \"c3579589-b0f5-4d12-998d-892f6e682086\") " pod="openshift-insights/insights-runtime-extractor-dj997" Jul 24 13:35:08.530269 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.530254 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsjxh\" (UniqueName: \"kubernetes.io/projected/005b211d-95c2-4052-972d-61ac571c313b-kube-api-access-jsjxh\") pod \"ingress-canary-zz9l9\" (UID: \"005b211d-95c2-4052-972d-61ac571c313b\") " pod="openshift-ingress-canary/ingress-canary-zz9l9" Jul 24 13:35:08.558131 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.558108 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dj997" Jul 24 13:35:08.580155 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.580131 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zz9l9" Jul 24 13:35:08.604012 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.603963 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:08.621335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.621301 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjgf4\" (UniqueName: \"kubernetes.io/projected/db90cec8-5114-4f7f-8966-79193d3cc584-kube-api-access-fjgf4\") pod \"dns-default-ck595\" (UID: \"db90cec8-5114-4f7f-8966-79193d3cc584\") " pod="openshift-dns/dns-default-ck595" Jul 24 13:35:08.621474 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.621377 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db90cec8-5114-4f7f-8966-79193d3cc584-metrics-tls\") pod \"dns-default-ck595\" (UID: \"db90cec8-5114-4f7f-8966-79193d3cc584\") " pod="openshift-dns/dns-default-ck595" Jul 24 13:35:08.621474 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.621443 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/db90cec8-5114-4f7f-8966-79193d3cc584-config-volume\") pod \"dns-default-ck595\" (UID: \"db90cec8-5114-4f7f-8966-79193d3cc584\") " pod="openshift-dns/dns-default-ck595" Jul 24 13:35:08.621599 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.621489 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db90cec8-5114-4f7f-8966-79193d3cc584-tmp-dir\") pod \"dns-default-ck595\" (UID: \"db90cec8-5114-4f7f-8966-79193d3cc584\") " pod="openshift-dns/dns-default-ck595" Jul 24 13:35:08.622467 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.621874 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db90cec8-5114-4f7f-8966-79193d3cc584-tmp-dir\") pod \"dns-default-ck595\" (UID: \"db90cec8-5114-4f7f-8966-79193d3cc584\") " pod="openshift-dns/dns-default-ck595" Jul 24 13:35:08.622467 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.622424 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/db90cec8-5114-4f7f-8966-79193d3cc584-config-volume\") pod \"dns-default-ck595\" (UID: \"db90cec8-5114-4f7f-8966-79193d3cc584\") " pod="openshift-dns/dns-default-ck595" Jul 24 13:35:08.625377 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.625357 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db90cec8-5114-4f7f-8966-79193d3cc584-metrics-tls\") pod \"dns-default-ck595\" (UID: \"db90cec8-5114-4f7f-8966-79193d3cc584\") " pod="openshift-dns/dns-default-ck595" Jul 24 13:35:08.635189 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.635163 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjgf4\" (UniqueName: \"kubernetes.io/projected/db90cec8-5114-4f7f-8966-79193d3cc584-kube-api-access-fjgf4\") pod \"dns-default-ck595\" (UID: \"db90cec8-5114-4f7f-8966-79193d3cc584\") " pod="openshift-dns/dns-default-ck595" Jul 24 13:35:08.642549 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.642277 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ck595" Jul 24 13:35:08.768020 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.767976 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c46b45557-8b5kp"] Jul 24 13:35:08.771758 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.771680 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dj997"] Jul 24 13:35:08.776330 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:35:08.776302 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3579589_b0f5_4d12_998d_892f6e682086.slice/crio-20ad55ebcafa3172fa104985bc8f1c6d80b81fe2efbd8cffdbaead94e4462782 WatchSource:0}: Error finding container 20ad55ebcafa3172fa104985bc8f1c6d80b81fe2efbd8cffdbaead94e4462782: Status 404 returned error can't find the container with id 20ad55ebcafa3172fa104985bc8f1c6d80b81fe2efbd8cffdbaead94e4462782 Jul 24 13:35:08.795029 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.794977 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zz9l9"] Jul 24 13:35:08.799300 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:35:08.799276 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod005b211d_95c2_4052_972d_61ac571c313b.slice/crio-109d21b990651ba14339f82ab25b54226f9ea1ccfbece1041008a8bdb34365a0 WatchSource:0}: Error finding container 109d21b990651ba14339f82ab25b54226f9ea1ccfbece1041008a8bdb34365a0: Status 404 returned error can't find the container with id 109d21b990651ba14339f82ab25b54226f9ea1ccfbece1041008a8bdb34365a0 Jul 24 13:35:08.837707 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:08.837682 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ck595"] Jul 24 13:35:09.414238 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:09.414204 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:35:09.414696 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:09.414669 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:35:09.415176 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:09.415153 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:35:09.435686 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:09.435503 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 13:35:09.435686 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:09.435518 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 13:35:09.435894 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:09.435845 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 13:35:09.436227 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:09.436075 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s8g8q\"" Jul 24 13:35:09.436227 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:09.436084 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 13:35:09.436377 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:09.436303 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gvb9z\"" Jul 24 13:35:09.774477 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:09.774395 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dj997" event={"ID":"c3579589-b0f5-4d12-998d-892f6e682086","Type":"ContainerStarted","Data":"20ad55ebcafa3172fa104985bc8f1c6d80b81fe2efbd8cffdbaead94e4462782"} Jul 24 13:35:09.778227 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:09.778196 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" event={"ID":"c1e0fc9a-b19b-4a76-aead-44c891fc755a","Type":"ContainerStarted","Data":"42fa70f487334703487bb41f0f41f0ab7907b24b14bec6baabe35e5502fbe45b"} Jul 24 13:35:09.778370 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:09.778247 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" event={"ID":"c1e0fc9a-b19b-4a76-aead-44c891fc755a","Type":"ContainerStarted","Data":"a818db5451110e4aad1832c0b7d27718498b1ded38585ca908c477f88544f75b"} Jul 24 13:35:09.778525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:09.778477 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:09.779954 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:09.779888 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ck595" event={"ID":"db90cec8-5114-4f7f-8966-79193d3cc584","Type":"ContainerStarted","Data":"38a820d692c5c7baf7249f82191d6f2413fda2abc560f36e4b565391bb8d9936"} Jul 24 13:35:09.781628 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:09.781601 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zz9l9" event={"ID":"005b211d-95c2-4052-972d-61ac571c313b","Type":"ContainerStarted","Data":"109d21b990651ba14339f82ab25b54226f9ea1ccfbece1041008a8bdb34365a0"} Jul 24 13:35:12.232762 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:12.232713 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:12.345818 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:12.345777 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:12.356544 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:12.356498 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" podStartSLOduration=5.356483505 podStartE2EDuration="5.356483505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:35:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:35:09.802894451 +0000 UTC m=+51.926276851" watchObservedRunningTime="2026-07-24 13:35:12.356483505 +0000 UTC m=+54.479865903" Jul 24 13:35:12.356693 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:12.356626 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9mr2c"] Jul 24 13:35:12.430584 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:12.430549 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9mr2c"] Jul 24 13:35:12.430753 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:12.430667 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9mr2c" Jul 24 13:35:12.435431 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:12.435403 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-hxhbv\"" Jul 24 13:35:12.435567 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:12.435439 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 24 13:35:12.459302 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:12.459267 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:12.461247 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:12.461224 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c5de7f2e-de24-42a5-9f53-6b7ef1f18343-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-9mr2c\" (UID: \"c5de7f2e-de24-42a5-9f53-6b7ef1f18343\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9mr2c" Jul 24 13:35:12.562362 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:12.562319 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c5de7f2e-de24-42a5-9f53-6b7ef1f18343-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-9mr2c\" (UID: \"c5de7f2e-de24-42a5-9f53-6b7ef1f18343\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9mr2c" Jul 24 13:35:12.562533 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:12.562440 2579 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 24 13:35:12.562584 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:12.562561 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5de7f2e-de24-42a5-9f53-6b7ef1f18343-tls-certificates podName:c5de7f2e-de24-42a5-9f53-6b7ef1f18343 nodeName:}" failed. No retries permitted until 2026-07-24 13:35:13.062538289 +0000 UTC m=+55.185920666 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/c5de7f2e-de24-42a5-9f53-6b7ef1f18343-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-9mr2c" (UID: "c5de7f2e-de24-42a5-9f53-6b7ef1f18343") : secret "prometheus-operator-admission-webhook-tls" not found Jul 24 13:35:12.900017 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:12.899922 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:13.028924 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:13.028883 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:13.066210 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:13.066177 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c5de7f2e-de24-42a5-9f53-6b7ef1f18343-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-9mr2c\" (UID: \"c5de7f2e-de24-42a5-9f53-6b7ef1f18343\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9mr2c" Jul 24 13:35:13.069045 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:13.069015 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c5de7f2e-de24-42a5-9f53-6b7ef1f18343-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-9mr2c\" (UID: \"c5de7f2e-de24-42a5-9f53-6b7ef1f18343\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9mr2c" Jul 24 13:35:13.229825 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:13.229701 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:13.329843 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:13.329795 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:13.341327 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:13.341293 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9mr2c" Jul 24 13:35:13.427475 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:13.427440 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:13.517364 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:13.517278 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:14.941850 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:14.941554 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9mr2c"] Jul 24 13:35:14.945647 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:35:14.945014 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5de7f2e_de24_42a5_9f53_6b7ef1f18343.slice/crio-7d052697cabc03826372d394fd57fccade3a811a99d0e7c1d1e680cc6e325a1c WatchSource:0}: Error finding container 7d052697cabc03826372d394fd57fccade3a811a99d0e7c1d1e680cc6e325a1c: Status 404 returned error can't find the container with id 7d052697cabc03826372d394fd57fccade3a811a99d0e7c1d1e680cc6e325a1c Jul 24 13:35:15.210071 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:15.210032 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:15.285859 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:15.285829 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:15.804049 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:15.803867 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ck595" event={"ID":"db90cec8-5114-4f7f-8966-79193d3cc584","Type":"ContainerStarted","Data":"7e2640d2b516cc4996b2b9147f41f69b1c6ff00cf477f043fbb32f757735a148"} Jul 24 13:35:15.804049 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:15.803960 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ck595" event={"ID":"db90cec8-5114-4f7f-8966-79193d3cc584","Type":"ContainerStarted","Data":"f7c0aab4622a5d1e880e5187a14dbd239854893198a7bdceae3be900cca9a247"} Jul 24 13:35:15.804049 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:15.804017 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ck595" Jul 24 13:35:15.805916 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:15.805883 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zz9l9" event={"ID":"005b211d-95c2-4052-972d-61ac571c313b","Type":"ContainerStarted","Data":"3dd4bd4dbbb7eb009efff3ded862230e892914e3681a4b1762fa74534586013d"} Jul 24 13:35:15.807341 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:15.807314 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9mr2c" event={"ID":"c5de7f2e-de24-42a5-9f53-6b7ef1f18343","Type":"ContainerStarted","Data":"7d052697cabc03826372d394fd57fccade3a811a99d0e7c1d1e680cc6e325a1c"} Jul 24 13:35:15.809478 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:15.809129 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dj997" event={"ID":"c3579589-b0f5-4d12-998d-892f6e682086","Type":"ContainerStarted","Data":"5c074021728bdf79bce0349b936190d03b4483aae8bbcddaa255ca602e479536"} Jul 24 13:35:15.828897 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:15.827951 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ck595" podStartSLOduration=1.892073322 podStartE2EDuration="7.827936731s" totalImagesPullingTime="5.935863409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:35:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:35:15.826466217 +0000 UTC m=+57.949848616" watchObservedRunningTime="2026-07-24 13:35:15.827936731 +0000 UTC m=+57.951319129" Jul 24 13:35:15.867240 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:15.865386 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:16.814917 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:16.814878 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9mr2c" event={"ID":"c5de7f2e-de24-42a5-9f53-6b7ef1f18343","Type":"ContainerStarted","Data":"93f9a42df82f835123822a8fb5c8f95f161c0af8ebf456a8fc9929c798cff1e7"} Jul 24 13:35:16.836085 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:16.836025 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9mr2c" podStartSLOduration=3.631406374 podStartE2EDuration="4.835979942s" totalImagesPullingTime="1.204573568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:35:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:35:16.835582427 +0000 UTC m=+58.958964827" watchObservedRunningTime="2026-07-24 13:35:16.835979942 +0000 UTC m=+58.959362340" Jul 24 13:35:16.836289 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:16.836182 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zz9l9" podStartSLOduration=2.845095685 podStartE2EDuration="8.836171251s" totalImagesPullingTime="5.991075566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:35:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:35:15.852354608 +0000 UTC m=+57.975737007" watchObservedRunningTime="2026-07-24 13:35:16.836171251 +0000 UTC m=+58.959553628" Jul 24 13:35:17.819831 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:17.819800 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9mr2c" Jul 24 13:35:17.825053 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:17.825027 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9mr2c" Jul 24 13:35:18.384653 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:18.384601 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:18.485003 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:18.484972 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:18.671014 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:18.670925 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:18.725738 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:18.725709 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-49x5m" Jul 24 13:35:18.917592 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:18.917563 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:19.018321 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:19.018229 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:19.829607 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:19.829565 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dj997" event={"ID":"c3579589-b0f5-4d12-998d-892f6e682086","Type":"ContainerStarted","Data":"e76d82b3c6ee817cb419262b397e9d36391cec0523f85a0e263abf1e9fa464d4"} Jul 24 13:35:20.572284 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:20.572251 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:20.673146 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:20.673104 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:20.749776 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:20.749731 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:22.789191 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:22.789095 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:22.841568 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:22.841538 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dj997" event={"ID":"c3579589-b0f5-4d12-998d-892f6e682086","Type":"ContainerStarted","Data":"1f8917410ba2e76b990d92a17e166b4cdcecd36360b33113cceaed9d9543cf4b"} Jul 24 13:35:22.873395 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:22.873350 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dj997" podStartSLOduration=1.440921178 podStartE2EDuration="14.873335889s" totalImagesPullingTime="13.432414711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:35:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:35:22.872057758 +0000 UTC m=+64.995440159" watchObservedRunningTime="2026-07-24 13:35:22.873335889 +0000 UTC m=+64.996718286" Jul 24 13:35:22.919040 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:22.918984 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:23.462032 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:23.461983 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:23.567970 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:23.567938 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:24.159024 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.158977 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs\") pod \"network-metrics-daemon-bsczm\" (UID: \"e2ed2aaf-08f3-4401-ba14-b742f3291961\") " pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:35:24.159420 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.159044 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret\") pod \"global-pull-secret-syncer-bxddb\" (UID: \"3fdc7954-a53e-440d-b8fa-889eaf069e9a\") " pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:35:24.162603 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.162580 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 13:35:24.162880 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.162864 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 13:35:24.172025 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.171977 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fdc7954-a53e-440d-b8fa-889eaf069e9a-original-pull-secret\") pod \"global-pull-secret-syncer-bxddb\" (UID: \"3fdc7954-a53e-440d-b8fa-889eaf069e9a\") " pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:35:24.172114 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.172084 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2ed2aaf-08f3-4401-ba14-b742f3291961-metrics-certs\") pod \"network-metrics-daemon-bsczm\" (UID: \"e2ed2aaf-08f3-4401-ba14-b742f3291961\") " pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:35:24.180113 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.180095 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s8g8q\"" Jul 24 13:35:24.185824 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.185809 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bxddb" Jul 24 13:35:24.187254 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.187240 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bsczm" Jul 24 13:35:24.260249 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.260212 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtnb4\" (UniqueName: \"kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4\") pod \"network-check-target-jkrfc\" (UID: \"3c327d99-2456-4a25-9ccf-47f75252aca9\") " pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:35:24.264762 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.264731 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 13:35:24.274157 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.274131 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 13:35:24.283676 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.283651 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtnb4\" (UniqueName: \"kubernetes.io/projected/3c327d99-2456-4a25-9ccf-47f75252aca9-kube-api-access-mtnb4\") pod \"network-check-target-jkrfc\" (UID: \"3c327d99-2456-4a25-9ccf-47f75252aca9\") " pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:35:24.327922 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.327893 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bsczm"] Jul 24 13:35:24.331180 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:35:24.331153 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2ed2aaf_08f3_4401_ba14_b742f3291961.slice/crio-2ec43e9d6871ec161a392dc073d2041ee68de5d587f8477f89c92ba92fb1303d WatchSource:0}: Error finding container 2ec43e9d6871ec161a392dc073d2041ee68de5d587f8477f89c92ba92fb1303d: Status 404 returned error can't find the container with id 2ec43e9d6871ec161a392dc073d2041ee68de5d587f8477f89c92ba92fb1303d Jul 24 13:35:24.342710 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.342690 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bxddb"] Jul 24 13:35:24.345747 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:35:24.345723 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fdc7954_a53e_440d_b8fa_889eaf069e9a.slice/crio-069f1a6dc06afb64c12e667251689db66cada761b335cd874a8fdfaf852e000a WatchSource:0}: Error finding container 069f1a6dc06afb64c12e667251689db66cada761b335cd874a8fdfaf852e000a: Status 404 returned error can't find the container with id 069f1a6dc06afb64c12e667251689db66cada761b335cd874a8fdfaf852e000a Jul 24 13:35:24.491216 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.491138 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gvb9z\"" Jul 24 13:35:24.498690 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.498673 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:35:24.625326 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.625295 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jkrfc"] Jul 24 13:35:24.628598 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:35:24.628559 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c327d99_2456_4a25_9ccf_47f75252aca9.slice/crio-2238c77ae5ef09e6a29a58391edfdbae1e004f2ef4ae462f33d78c6094916d5b WatchSource:0}: Error finding container 2238c77ae5ef09e6a29a58391edfdbae1e004f2ef4ae462f33d78c6094916d5b: Status 404 returned error can't find the container with id 2238c77ae5ef09e6a29a58391edfdbae1e004f2ef4ae462f33d78c6094916d5b Jul 24 13:35:24.854349 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.854313 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bsczm" event={"ID":"e2ed2aaf-08f3-4401-ba14-b742f3291961","Type":"ContainerStarted","Data":"2ec43e9d6871ec161a392dc073d2041ee68de5d587f8477f89c92ba92fb1303d"} Jul 24 13:35:24.855794 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.855758 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jkrfc" event={"ID":"3c327d99-2456-4a25-9ccf-47f75252aca9","Type":"ContainerStarted","Data":"2238c77ae5ef09e6a29a58391edfdbae1e004f2ef4ae462f33d78c6094916d5b"} Jul 24 13:35:24.857052 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:24.857020 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bxddb" event={"ID":"3fdc7954-a53e-440d-b8fa-889eaf069e9a","Type":"ContainerStarted","Data":"069f1a6dc06afb64c12e667251689db66cada761b335cd874a8fdfaf852e000a"} Jul 24 13:35:25.576849 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.576812 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:25.783859 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.783816 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:25.863031 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.862927 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bsczm" event={"ID":"e2ed2aaf-08f3-4401-ba14-b742f3291961","Type":"ContainerStarted","Data":"54f0bc137ff5af623158e7a8a21f2817dcfea8710123cd5f71106f368dd94573"} Jul 24 13:35:25.863207 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.863056 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bsczm" event={"ID":"e2ed2aaf-08f3-4401-ba14-b742f3291961","Type":"ContainerStarted","Data":"7f5169829d21f8ffe26eb44325438eb57711d3cfdc0e6a5c17a023a1221b12b7"} Jul 24 13:35:25.877778 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.877745 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb"] Jul 24 13:35:25.903645 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.901287 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb"] Jul 24 13:35:25.903645 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.901539 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bsczm" podStartSLOduration=66.975165047 podStartE2EDuration="1m7.90148459s" totalImagesPullingTime="926.319543ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:35:25.900373945 +0000 UTC m=+68.023756345" watchObservedRunningTime="2026-07-24 13:35:25.90148459 +0000 UTC m=+68.024866989" Jul 24 13:35:25.903645 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.902419 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:25.903942 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.903689 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-db65f"] Jul 24 13:35:25.906673 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.906226 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 24 13:35:25.906673 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.906316 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:25.906673 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.906533 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 24 13:35:25.906673 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.906624 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 24 13:35:25.907145 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.906869 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 24 13:35:25.907145 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.907086 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 24 13:35:25.907145 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.907112 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-h4z8m\"" Jul 24 13:35:25.907480 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.907454 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 24 13:35:25.917141 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.917116 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:25.921139 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.920555 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 24 13:35:25.921139 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.920850 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 24 13:35:25.921139 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.921038 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mwmx5\"" Jul 24 13:35:25.922961 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.922808 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 24 13:35:25.973291 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.973256 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1fcd557-e028-474d-8114-1c3d52016712-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:25.973467 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.973304 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1fcd557-e028-474d-8114-1c3d52016712-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:25.973467 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.973335 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c33fa643-8a39-47d5-bdfc-2ed9529413d4-node-exporter-tls\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:25.973467 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.973365 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c33fa643-8a39-47d5-bdfc-2ed9529413d4-metrics-client-ca\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:25.973467 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.973423 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c33fa643-8a39-47d5-bdfc-2ed9529413d4-sys\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:25.973675 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.973475 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c33fa643-8a39-47d5-bdfc-2ed9529413d4-node-exporter-accelerators-collector-config\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:25.973675 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.973496 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4s8h\" (UniqueName: \"kubernetes.io/projected/c33fa643-8a39-47d5-bdfc-2ed9529413d4-kube-api-access-v4s8h\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:25.973675 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.973515 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1fcd557-e028-474d-8114-1c3d52016712-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:25.973675 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.973555 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a1fcd557-e028-474d-8114-1c3d52016712-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:25.973675 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.973601 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c33fa643-8a39-47d5-bdfc-2ed9529413d4-node-exporter-textfile\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:25.973675 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.973652 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c33fa643-8a39-47d5-bdfc-2ed9529413d4-node-exporter-wtmp\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:25.973675 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.973678 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwsq5\" (UniqueName: \"kubernetes.io/projected/a1fcd557-e028-474d-8114-1c3d52016712-kube-api-access-vwsq5\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:25.973937 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.973704 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c33fa643-8a39-47d5-bdfc-2ed9529413d4-root\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:25.973937 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.973730 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a1fcd557-e028-474d-8114-1c3d52016712-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:25.973937 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:25.973751 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c33fa643-8a39-47d5-bdfc-2ed9529413d4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.063306 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.063263 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:26.074481 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.074449 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a1fcd557-e028-474d-8114-1c3d52016712-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:26.074616 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.074492 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c33fa643-8a39-47d5-bdfc-2ed9529413d4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.074616 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.074529 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1fcd557-e028-474d-8114-1c3d52016712-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:26.074616 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.074560 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1fcd557-e028-474d-8114-1c3d52016712-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:26.074616 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.074586 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c33fa643-8a39-47d5-bdfc-2ed9529413d4-node-exporter-tls\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.074616 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.074612 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c33fa643-8a39-47d5-bdfc-2ed9529413d4-metrics-client-ca\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.074863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.074639 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c33fa643-8a39-47d5-bdfc-2ed9529413d4-sys\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.074863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.074670 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c33fa643-8a39-47d5-bdfc-2ed9529413d4-node-exporter-accelerators-collector-config\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.074863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.074694 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4s8h\" (UniqueName: \"kubernetes.io/projected/c33fa643-8a39-47d5-bdfc-2ed9529413d4-kube-api-access-v4s8h\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.074863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.074726 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1fcd557-e028-474d-8114-1c3d52016712-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:26.074863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.074754 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a1fcd557-e028-474d-8114-1c3d52016712-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:26.074863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.074787 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c33fa643-8a39-47d5-bdfc-2ed9529413d4-node-exporter-textfile\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.074863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.074822 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c33fa643-8a39-47d5-bdfc-2ed9529413d4-node-exporter-wtmp\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.074863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.074851 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwsq5\" (UniqueName: \"kubernetes.io/projected/a1fcd557-e028-474d-8114-1c3d52016712-kube-api-access-vwsq5\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:26.075148 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.074883 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c33fa643-8a39-47d5-bdfc-2ed9529413d4-root\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.075148 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.074966 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c33fa643-8a39-47d5-bdfc-2ed9529413d4-root\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.075289 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.075252 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a1fcd557-e028-474d-8114-1c3d52016712-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:26.075350 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.075323 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1fcd557-e028-474d-8114-1c3d52016712-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:26.075399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.075356 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c33fa643-8a39-47d5-bdfc-2ed9529413d4-node-exporter-accelerators-collector-config\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.075454 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:26.075437 2579 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 24 13:35:26.075522 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:26.075494 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1fcd557-e028-474d-8114-1c3d52016712-kube-state-metrics-tls podName:a1fcd557-e028-474d-8114-1c3d52016712 nodeName:}" failed. No retries permitted until 2026-07-24 13:35:26.575473852 +0000 UTC m=+68.698856232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a1fcd557-e028-474d-8114-1c3d52016712-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-zgvfb" (UID: "a1fcd557-e028-474d-8114-1c3d52016712") : secret "kube-state-metrics-tls" not found Jul 24 13:35:26.075592 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:26.075567 2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 24 13:35:26.075651 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:26.075602 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c33fa643-8a39-47d5-bdfc-2ed9529413d4-node-exporter-tls podName:c33fa643-8a39-47d5-bdfc-2ed9529413d4 nodeName:}" failed. No retries permitted until 2026-07-24 13:35:26.575589582 +0000 UTC m=+68.698971964 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c33fa643-8a39-47d5-bdfc-2ed9529413d4-node-exporter-tls") pod "node-exporter-db65f" (UID: "c33fa643-8a39-47d5-bdfc-2ed9529413d4") : secret "node-exporter-tls" not found Jul 24 13:35:26.075651 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.075642 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c33fa643-8a39-47d5-bdfc-2ed9529413d4-sys\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.075759 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.075644 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a1fcd557-e028-474d-8114-1c3d52016712-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:26.075813 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.075774 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c33fa643-8a39-47d5-bdfc-2ed9529413d4-node-exporter-wtmp\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.075813 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.075791 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c33fa643-8a39-47d5-bdfc-2ed9529413d4-node-exporter-textfile\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.076179 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.076143 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c33fa643-8a39-47d5-bdfc-2ed9529413d4-metrics-client-ca\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.078127 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.078106 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1fcd557-e028-474d-8114-1c3d52016712-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:26.078319 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.078299 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c33fa643-8a39-47d5-bdfc-2ed9529413d4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.093280 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.093240 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwsq5\" (UniqueName: \"kubernetes.io/projected/a1fcd557-e028-474d-8114-1c3d52016712-kube-api-access-vwsq5\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:26.094317 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.094290 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4s8h\" (UniqueName: \"kubernetes.io/projected/c33fa643-8a39-47d5-bdfc-2ed9529413d4-kube-api-access-v4s8h\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.169761 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.169659 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:26.373917 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.373883 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:26.476413 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.476297 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:26.580011 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.579448 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1fcd557-e028-474d-8114-1c3d52016712-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:26.580011 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.579494 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c33fa643-8a39-47d5-bdfc-2ed9529413d4-node-exporter-tls\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.582576 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.582538 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c33fa643-8a39-47d5-bdfc-2ed9529413d4-node-exporter-tls\") pod \"node-exporter-db65f\" (UID: \"c33fa643-8a39-47d5-bdfc-2ed9529413d4\") " pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.583540 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.583512 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:26.586101 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.586073 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1fcd557-e028-474d-8114-1c3d52016712-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-zgvfb\" (UID: \"a1fcd557-e028-474d-8114-1c3d52016712\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:26.814651 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.814599 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" Jul 24 13:35:26.821982 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.821952 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ck595" Jul 24 13:35:26.827730 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.827704 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-db65f" Jul 24 13:35:26.835511 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.835477 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:26.939223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.939091 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:26.952634 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.950956 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 13:35:26.976871 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.976547 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 13:35:26.976871 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.976782 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:26.982115 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.981383 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 24 13:35:26.982115 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.981554 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 24 13:35:26.982115 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.981753 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 24 13:35:26.982115 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.981834 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 24 13:35:26.982115 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.981901 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 24 13:35:26.982115 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.981982 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 24 13:35:26.982115 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.982035 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 24 13:35:26.984309 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.983346 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 24 13:35:26.984309 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.983605 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tbmt9\"" Jul 24 13:35:26.984309 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:26.983850 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 24 13:35:27.023134 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.023101 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb"] Jul 24 13:35:27.026934 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:35:27.026891 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1fcd557_e028_474d_8114_1c3d52016712.slice/crio-81ca1ac9bf170f15af431640fc72ea7ce583e0ca3361f1ac7f51f4c2a240644d WatchSource:0}: Error finding container 81ca1ac9bf170f15af431640fc72ea7ce583e0ca3361f1ac7f51f4c2a240644d: Status 404 returned error can't find the container with id 81ca1ac9bf170f15af431640fc72ea7ce583e0ca3361f1ac7f51f4c2a240644d Jul 24 13:35:27.084075 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.084003 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.084075 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.084045 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/955bb86f-1d3e-4ae1-9b25-8725023194a4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.084276 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.084105 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/955bb86f-1d3e-4ae1-9b25-8725023194a4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.084276 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.084150 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.084276 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.084195 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.084276 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.084222 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.084276 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.084255 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.084453 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.084338 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-web-config\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.084453 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.084372 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxq58\" (UniqueName: \"kubernetes.io/projected/955bb86f-1d3e-4ae1-9b25-8725023194a4-kube-api-access-kxq58\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.084453 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.084399 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/955bb86f-1d3e-4ae1-9b25-8725023194a4-config-out\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.084453 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.084416 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/955bb86f-1d3e-4ae1-9b25-8725023194a4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.084453 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.084442 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-config-volume\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.084627 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.084461 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/955bb86f-1d3e-4ae1-9b25-8725023194a4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.173685 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.173650 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:27.185601 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.185577 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.185746 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.185617 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.185746 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.185643 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.185746 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.185663 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.185746 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.185702 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-web-config\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.185746 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.185727 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxq58\" (UniqueName: \"kubernetes.io/projected/955bb86f-1d3e-4ae1-9b25-8725023194a4-kube-api-access-kxq58\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.186008 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.185810 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/955bb86f-1d3e-4ae1-9b25-8725023194a4-config-out\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.186008 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.185836 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/955bb86f-1d3e-4ae1-9b25-8725023194a4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.186008 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.185865 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-config-volume\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.186008 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.185887 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/955bb86f-1d3e-4ae1-9b25-8725023194a4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.186008 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.185925 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.186008 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.185950 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/955bb86f-1d3e-4ae1-9b25-8725023194a4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.186008 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.185965 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/955bb86f-1d3e-4ae1-9b25-8725023194a4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.186749 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.186722 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/955bb86f-1d3e-4ae1-9b25-8725023194a4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.188174 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.187018 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/955bb86f-1d3e-4ae1-9b25-8725023194a4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.188174 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.188114 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/955bb86f-1d3e-4ae1-9b25-8725023194a4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.189269 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.189227 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.189686 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.189667 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.190013 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.189963 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-config-volume\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.190298 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.190236 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-web-config\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.190719 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.190696 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/955bb86f-1d3e-4ae1-9b25-8725023194a4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.190719 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.190712 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.190913 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.190882 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.190977 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.190945 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/955bb86f-1d3e-4ae1-9b25-8725023194a4-config-out\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.191722 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.191698 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.198411 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.198392 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxq58\" (UniqueName: \"kubernetes.io/projected/955bb86f-1d3e-4ae1-9b25-8725023194a4-kube-api-access-kxq58\") pod \"alertmanager-main-0\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.292761 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.292715 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:35:27.873857 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.873815 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" event={"ID":"a1fcd557-e028-474d-8114-1c3d52016712","Type":"ContainerStarted","Data":"81ca1ac9bf170f15af431640fc72ea7ce583e0ca3361f1ac7f51f4c2a240644d"} Jul 24 13:35:27.875146 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:27.875114 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-db65f" event={"ID":"c33fa643-8a39-47d5-bdfc-2ed9529413d4","Type":"ContainerStarted","Data":"53aa7315912e291d086511855dab427761edf8459929fb3c62322f0b2ae796b9"} Jul 24 13:35:28.532749 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:28.532684 2579 patch_prober.go:28] interesting pod/image-registry-6c46b45557-8b5kp container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 24 13:35:28.532939 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:28.532773 2579 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" podUID="c1e0fc9a-b19b-4a76-aead-44c891fc755a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 13:35:28.926441 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:28.926400 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:29.028133 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:29.028092 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:29.367900 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:29.367863 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:29.678678 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:29.678601 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:30.463907 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.463423 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-698d4d8b68-bzxrc"] Jul 24 13:35:30.467684 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.467659 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.472634 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.472306 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 24 13:35:30.477138 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.475375 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-5qbsubrte0tss\"" Jul 24 13:35:30.477138 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.475663 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 24 13:35:30.477138 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.475874 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 24 13:35:30.477138 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.476083 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-rtnf5\"" Jul 24 13:35:30.477138 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.476323 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 24 13:35:30.479830 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.479718 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 13:35:30.485456 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.485436 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-698d4d8b68-bzxrc"] Jul 24 13:35:30.492510 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:35:30.489842 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod955bb86f_1d3e_4ae1_9b25_8725023194a4.slice/crio-494312f9a10c6ed03273402576ac835af04bd9b7ad3fd8f6350c71082bf7e9ef WatchSource:0}: Error finding container 494312f9a10c6ed03273402576ac835af04bd9b7ad3fd8f6350c71082bf7e9ef: Status 404 returned error can't find the container with id 494312f9a10c6ed03273402576ac835af04bd9b7ad3fd8f6350c71082bf7e9ef Jul 24 13:35:30.516857 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.516817 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bbef461c-8313-4bea-8810-8fe7340cb033-audit-log\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.516985 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.516913 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bbef461c-8313-4bea-8810-8fe7340cb033-client-ca-bundle\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.516985 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.516975 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmx26\" (UniqueName: \"kubernetes.io/projected/bbef461c-8313-4bea-8810-8fe7340cb033-kube-api-access-mmx26\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.516985 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.517029 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbef461c-8313-4bea-8810-8fe7340cb033-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.517224 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.517083 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bbef461c-8313-4bea-8810-8fe7340cb033-secret-metrics-server-client-certs\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.517224 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.517135 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bbef461c-8313-4bea-8810-8fe7340cb033-secret-metrics-server-tls\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.517224 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.517171 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bbef461c-8313-4bea-8810-8fe7340cb033-metrics-server-audit-profiles\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.618402 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.618343 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bbef461c-8313-4bea-8810-8fe7340cb033-metrics-server-audit-profiles\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.618583 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.618425 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bbef461c-8313-4bea-8810-8fe7340cb033-audit-log\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.618583 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.618457 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bbef461c-8313-4bea-8810-8fe7340cb033-client-ca-bundle\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.618583 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.618485 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmx26\" (UniqueName: \"kubernetes.io/projected/bbef461c-8313-4bea-8810-8fe7340cb033-kube-api-access-mmx26\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.618583 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.618516 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbef461c-8313-4bea-8810-8fe7340cb033-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.618583 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.618564 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bbef461c-8313-4bea-8810-8fe7340cb033-secret-metrics-server-client-certs\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.618843 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.618609 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bbef461c-8313-4bea-8810-8fe7340cb033-secret-metrics-server-tls\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.619567 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.619536 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bbef461c-8313-4bea-8810-8fe7340cb033-metrics-server-audit-profiles\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.620252 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.620201 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bbef461c-8313-4bea-8810-8fe7340cb033-audit-log\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.620693 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.620667 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbef461c-8313-4bea-8810-8fe7340cb033-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.622404 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.622381 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bbef461c-8313-4bea-8810-8fe7340cb033-secret-metrics-server-tls\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.622612 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.622597 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bbef461c-8313-4bea-8810-8fe7340cb033-client-ca-bundle\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.622799 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.622779 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bbef461c-8313-4bea-8810-8fe7340cb033-secret-metrics-server-client-certs\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.639736 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.639715 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmx26\" (UniqueName: \"kubernetes.io/projected/bbef461c-8313-4bea-8810-8fe7340cb033-kube-api-access-mmx26\") pod \"metrics-server-698d4d8b68-bzxrc\" (UID: \"bbef461c-8313-4bea-8810-8fe7340cb033\") " pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.651475 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.651449 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:30.677254 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.677229 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-vnbmn"] Jul 24 13:35:30.681468 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.681451 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vnbmn" Jul 24 13:35:30.684909 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.684891 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 24 13:35:30.685857 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.685843 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2wqxv\"" Jul 24 13:35:30.718176 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.718135 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-vnbmn"] Jul 24 13:35:30.790323 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.790294 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c46b45557-8b5kp" Jul 24 13:35:30.791479 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.791449 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:30.793387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.793368 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:30.820233 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.820152 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fdd282a6-b290-4a78-9a81-d078d731c942-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-vnbmn\" (UID: \"fdd282a6-b290-4a78-9a81-d078d731c942\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vnbmn" Jul 24 13:35:30.820233 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.820196 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6fzh\" (UniqueName: \"kubernetes.io/projected/fdd282a6-b290-4a78-9a81-d078d731c942-kube-api-access-z6fzh\") pod \"monitoring-plugin-68b7fc9574-vnbmn\" (UID: \"fdd282a6-b290-4a78-9a81-d078d731c942\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vnbmn" Jul 24 13:35:30.890110 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.890070 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jkrfc" event={"ID":"3c327d99-2456-4a25-9ccf-47f75252aca9","Type":"ContainerStarted","Data":"818bccab00cccfbda5e322298d0d3ae0aab2d4387ea88a0b78d4006ced4b0c7a"} Jul 24 13:35:30.890346 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.890328 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:35:30.892006 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.891941 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bxddb" event={"ID":"3fdc7954-a53e-440d-b8fa-889eaf069e9a","Type":"ContainerStarted","Data":"04b4cc7dbaf834ab59b96277107fc506e19954c56d2b1db56fa4adf0421bea53"} Jul 24 13:35:30.894336 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.894228 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955bb86f-1d3e-4ae1-9b25-8725023194a4","Type":"ContainerStarted","Data":"494312f9a10c6ed03273402576ac835af04bd9b7ad3fd8f6350c71082bf7e9ef"} Jul 24 13:35:30.897205 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.897175 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" event={"ID":"a1fcd557-e028-474d-8114-1c3d52016712","Type":"ContainerStarted","Data":"fb6d9502b86823356d91ffe5015407ac3b3f54382fcc231a8204369efbf6e728"} Jul 24 13:35:30.897311 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.897237 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" event={"ID":"a1fcd557-e028-474d-8114-1c3d52016712","Type":"ContainerStarted","Data":"35ea86929b0f57f5fa6da63b764dcdb9e297df1d6b2a485a0b28f963c48d8c78"} Jul 24 13:35:30.897311 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.897267 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" event={"ID":"a1fcd557-e028-474d-8114-1c3d52016712","Type":"ContainerStarted","Data":"1fb92596fa9e8b2b42a3c0ab60039a184daba96a1eb2aed1bbae4d3ca327d5d4"} Jul 24 13:35:30.901111 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.899688 2579 generic.go:356] "Generic (PLEG): container finished" podID="c33fa643-8a39-47d5-bdfc-2ed9529413d4" containerID="0ad4368fb8899b2d32ec5ba59dd9963c58e9600c139337115c0bba00c59450d1" exitCode=0 Jul 24 13:35:30.901111 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.899731 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-db65f" event={"ID":"c33fa643-8a39-47d5-bdfc-2ed9529413d4","Type":"ContainerDied","Data":"0ad4368fb8899b2d32ec5ba59dd9963c58e9600c139337115c0bba00c59450d1"} Jul 24 13:35:30.905855 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.905776 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:30.922328 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.921179 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fdd282a6-b290-4a78-9a81-d078d731c942-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-vnbmn\" (UID: \"fdd282a6-b290-4a78-9a81-d078d731c942\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vnbmn" Jul 24 13:35:30.922328 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.921250 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6fzh\" (UniqueName: \"kubernetes.io/projected/fdd282a6-b290-4a78-9a81-d078d731c942-kube-api-access-z6fzh\") pod \"monitoring-plugin-68b7fc9574-vnbmn\" (UID: \"fdd282a6-b290-4a78-9a81-d078d731c942\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vnbmn" Jul 24 13:35:30.922328 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:30.922081 2579 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 24 13:35:30.922328 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:35:30.922142 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdd282a6-b290-4a78-9a81-d078d731c942-monitoring-plugin-cert podName:fdd282a6-b290-4a78-9a81-d078d731c942 nodeName:}" failed. No retries permitted until 2026-07-24 13:35:31.422121754 +0000 UTC m=+73.545504135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/fdd282a6-b290-4a78-9a81-d078d731c942-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-vnbmn" (UID: "fdd282a6-b290-4a78-9a81-d078d731c942") : secret "monitoring-plugin-cert" not found Jul 24 13:35:30.927410 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.927242 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jkrfc" podStartSLOduration=67.278450021 podStartE2EDuration="1m12.927224185s" totalImagesPullingTime="5.648774164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:35:30.925774676 +0000 UTC m=+73.049157074" watchObservedRunningTime="2026-07-24 13:35:30.927224185 +0000 UTC m=+73.050606585" Jul 24 13:35:30.936109 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.936080 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6fzh\" (UniqueName: \"kubernetes.io/projected/fdd282a6-b290-4a78-9a81-d078d731c942-kube-api-access-z6fzh\") pod \"monitoring-plugin-68b7fc9574-vnbmn\" (UID: \"fdd282a6-b290-4a78-9a81-d078d731c942\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vnbmn" Jul 24 13:35:30.968516 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.968443 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bxddb" podStartSLOduration=67.036668334 podStartE2EDuration="1m12.968418073s" totalImagesPullingTime="5.931749739s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:35:30.954140623 +0000 UTC m=+73.077523022" watchObservedRunningTime="2026-07-24 13:35:30.968418073 +0000 UTC m=+73.091800472" Jul 24 13:35:30.972891 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.970403 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-698d4d8b68-bzxrc"] Jul 24 13:35:30.979153 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:35:30.979118 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbef461c_8313_4bea_8810_8fe7340cb033.slice/crio-dafbc83e6d86494b2d9cae9e8d8395e26512751784720841c6babe1fe5e8bcdf WatchSource:0}: Error finding container dafbc83e6d86494b2d9cae9e8d8395e26512751784720841c6babe1fe5e8bcdf: Status 404 returned error can't find the container with id dafbc83e6d86494b2d9cae9e8d8395e26512751784720841c6babe1fe5e8bcdf Jul 24 13:35:30.982898 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:30.982852 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zgvfb" podStartSLOduration=2.731755435 podStartE2EDuration="5.982836046s" totalImagesPullingTime="3.251080611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:35:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:35:30.981453245 +0000 UTC m=+73.104835644" watchObservedRunningTime="2026-07-24 13:35:30.982836046 +0000 UTC m=+73.106218444" Jul 24 13:35:31.024474 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:31.024440 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:31.109066 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:31.108962 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:31.414691 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:31.414582 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:31.425493 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:31.425465 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fdd282a6-b290-4a78-9a81-d078d731c942-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-vnbmn\" (UID: \"fdd282a6-b290-4a78-9a81-d078d731c942\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vnbmn" Jul 24 13:35:31.428150 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:31.428122 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fdd282a6-b290-4a78-9a81-d078d731c942-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-vnbmn\" (UID: \"fdd282a6-b290-4a78-9a81-d078d731c942\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vnbmn" Jul 24 13:35:31.515260 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:31.515211 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:31.591300 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:31.591269 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vnbmn" Jul 24 13:35:31.743779 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:31.743749 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-vnbmn"] Jul 24 13:35:31.787967 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:35:31.787926 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdd282a6_b290_4a78_9a81_d078d731c942.slice/crio-bea695adfcae7f89c2bf241d6db00c4c90b80818dee6c6d60ba54c4e895a8cfd WatchSource:0}: Error finding container bea695adfcae7f89c2bf241d6db00c4c90b80818dee6c6d60ba54c4e895a8cfd: Status 404 returned error can't find the container with id bea695adfcae7f89c2bf241d6db00c4c90b80818dee6c6d60ba54c4e895a8cfd Jul 24 13:35:31.906045 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:31.906008 2579 generic.go:356] "Generic (PLEG): container finished" podID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerID="bee6d482dff4bc989747b904bb435bd85c860ca3bf3c70a0d63a135f13f06398" exitCode=0 Jul 24 13:35:31.906229 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:31.906092 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955bb86f-1d3e-4ae1-9b25-8725023194a4","Type":"ContainerDied","Data":"bee6d482dff4bc989747b904bb435bd85c860ca3bf3c70a0d63a135f13f06398"} Jul 24 13:35:31.909180 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:31.909152 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-db65f" event={"ID":"c33fa643-8a39-47d5-bdfc-2ed9529413d4","Type":"ContainerStarted","Data":"006361f6a00542598837fe39b1167527e39cf9651bcfd7c911501ca612fed53c"} Jul 24 13:35:31.909309 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:31.909207 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-db65f" event={"ID":"c33fa643-8a39-47d5-bdfc-2ed9529413d4","Type":"ContainerStarted","Data":"22f354780a7cc34493be865e92047db7a69ac1f889072c0d1a461d4513476b86"} Jul 24 13:35:31.910619 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:31.910593 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vnbmn" event={"ID":"fdd282a6-b290-4a78-9a81-d078d731c942","Type":"ContainerStarted","Data":"bea695adfcae7f89c2bf241d6db00c4c90b80818dee6c6d60ba54c4e895a8cfd"} Jul 24 13:35:31.911949 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:31.911922 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" event={"ID":"bbef461c-8313-4bea-8810-8fe7340cb033","Type":"ContainerStarted","Data":"dafbc83e6d86494b2d9cae9e8d8395e26512751784720841c6babe1fe5e8bcdf"} Jul 24 13:35:31.976185 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:31.976066 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-db65f" podStartSLOduration=3.462719866 podStartE2EDuration="6.976045444s" totalImagesPullingTime="3.396898667s" totalInitContainerRuntime="116.426911ms" isStatefulPod=true podCreationTimestamp="2026-07-24 13:35:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:35:31.974529863 +0000 UTC m=+74.097912262" watchObservedRunningTime="2026-07-24 13:35:31.976045444 +0000 UTC m=+74.099427842" Jul 24 13:35:32.265213 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:32.265134 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:32.386038 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:32.385982 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:32.794360 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:32.794327 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:32.899190 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:32.899144 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:32.918624 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:32.918585 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" event={"ID":"bbef461c-8313-4bea-8810-8fe7340cb033","Type":"ContainerStarted","Data":"e44f848acfdf772825f5d8128d3aa022be1ad1b512326c2f971889df87c83615"} Jul 24 13:35:32.943288 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:32.943221 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" podStartSLOduration=1.14125455 podStartE2EDuration="2.943198905s" totalImagesPullingTime="1.801944355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:35:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:35:32.942545411 +0000 UTC m=+75.065927820" watchObservedRunningTime="2026-07-24 13:35:32.943198905 +0000 UTC m=+75.066581305" Jul 24 13:35:33.924059 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:33.924025 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vnbmn" event={"ID":"fdd282a6-b290-4a78-9a81-d078d731c942","Type":"ContainerStarted","Data":"d7f2d298a1ff5650ae536b456db979bbdf7e036d8a9e78dfc1f1142104c34926"} Jul 24 13:35:33.944077 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:33.943974 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vnbmn" podStartSLOduration=2.341827684 podStartE2EDuration="3.94395765s" totalImagesPullingTime="1.602129966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:35:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:35:33.943292485 +0000 UTC m=+76.066674882" watchObservedRunningTime="2026-07-24 13:35:33.94395765 +0000 UTC m=+76.067340047" Jul 24 13:35:34.932290 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:34.932249 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955bb86f-1d3e-4ae1-9b25-8725023194a4","Type":"ContainerStarted","Data":"c2bdf763884ced2f7c02b0440247ffcd71b0a1e505ae6d16f2d0f676fdbe067f"} Jul 24 13:35:34.932750 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:34.932355 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955bb86f-1d3e-4ae1-9b25-8725023194a4","Type":"ContainerStarted","Data":"89d0bc92dffa8ed09a19df5b8ac83538fe13fdc37b198cff95c69b59605daa9c"} Jul 24 13:35:34.932750 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:34.932401 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955bb86f-1d3e-4ae1-9b25-8725023194a4","Type":"ContainerStarted","Data":"8596d554b27af0a1b4c4e24853e9851ffd9d9cf2fc6fbdb630d0312b9c78e45c"} Jul 24 13:35:34.932750 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:34.932443 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955bb86f-1d3e-4ae1-9b25-8725023194a4","Type":"ContainerStarted","Data":"d139208254fbc85d3e9dfc1c233100ed98f80f826465b35e468e77cc97b9db66"} Jul 24 13:35:34.932750 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:34.932482 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955bb86f-1d3e-4ae1-9b25-8725023194a4","Type":"ContainerStarted","Data":"073215642ba4662fb18080db20a4009c173356a800fc0da8c4351082a7d00167"} Jul 24 13:35:34.932750 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:34.932669 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vnbmn" Jul 24 13:35:34.938717 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:34.938692 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vnbmn" Jul 24 13:35:35.199446 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:35.199322 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:35.285015 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:35.284940 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:35.708962 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:35.708921 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:35.784832 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:35.784795 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:35.943918 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:35.943876 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955bb86f-1d3e-4ae1-9b25-8725023194a4","Type":"ContainerStarted","Data":"8d24d5147f1e712f545910a545b7f2a94dee6048921f821ea2152050b5d4d367"} Jul 24 13:35:36.270101 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:36.270058 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:36.354141 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:36.354103 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:37.759239 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:37.759206 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:37.922906 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:37.922875 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:37.997880 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:37.997840 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:38.793601 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:38.793486 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:38.905933 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:38.905892 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:41.298107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:41.298066 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:41.900299 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:41.900263 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:42.022330 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:42.022290 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:42.118090 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:42.118047 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:44.409315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:44.409283 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:44.515013 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:44.514966 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:45.244169 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:45.244126 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:45.344915 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:45.344883 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:50.793647 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:50.793612 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:35:50.794107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:35:50.793666 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:36:00.242367 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:00.242328 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:00.322838 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:00.322807 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:01.914143 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:01.914104 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jkrfc" Jul 24 13:36:01.946668 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:01.946614 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=31.970701606 podStartE2EDuration="35.946593011s" totalImagesPullingTime="3.956804345s" totalInitContainerRuntime="19.08706ms" isStatefulPod=false podCreationTimestamp="2026-07-24 13:35:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:35:35.981640009 +0000 UTC m=+78.105022406" watchObservedRunningTime="2026-07-24 13:36:01.946593011 +0000 UTC m=+104.069975471" Jul 24 13:36:10.804420 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:10.804390 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:36:10.808290 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:10.808269 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-698d4d8b68-bzxrc" Jul 24 13:36:14.304656 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:14.304622 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:14.379689 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:14.379649 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:45.336682 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:45.336648 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:45.534080 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:45.534044 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:46.398070 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:46.398036 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 13:36:46.398436 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:46.398330 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="alertmanager" containerID="cri-o://073215642ba4662fb18080db20a4009c173356a800fc0da8c4351082a7d00167" gracePeriod=120 Jul 24 13:36:46.398436 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:46.398401 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="kube-rbac-proxy-metric" containerID="cri-o://c2bdf763884ced2f7c02b0440247ffcd71b0a1e505ae6d16f2d0f676fdbe067f" gracePeriod=120 Jul 24 13:36:46.398612 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:46.398438 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="kube-rbac-proxy-web" containerID="cri-o://8596d554b27af0a1b4c4e24853e9851ffd9d9cf2fc6fbdb630d0312b9c78e45c" gracePeriod=120 Jul 24 13:36:46.398612 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:46.398465 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="prom-label-proxy" containerID="cri-o://8d24d5147f1e712f545910a545b7f2a94dee6048921f821ea2152050b5d4d367" gracePeriod=120 Jul 24 13:36:46.398709 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:46.398457 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="config-reloader" containerID="cri-o://d139208254fbc85d3e9dfc1c233100ed98f80f826465b35e468e77cc97b9db66" gracePeriod=120 Jul 24 13:36:46.398770 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:46.398466 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="kube-rbac-proxy" containerID="cri-o://89d0bc92dffa8ed09a19df5b8ac83538fe13fdc37b198cff95c69b59605daa9c" gracePeriod=120 Jul 24 13:36:47.211889 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.211860 2579 generic.go:356] "Generic (PLEG): container finished" podID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerID="8d24d5147f1e712f545910a545b7f2a94dee6048921f821ea2152050b5d4d367" exitCode=0 Jul 24 13:36:47.211889 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.211884 2579 generic.go:356] "Generic (PLEG): container finished" podID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerID="89d0bc92dffa8ed09a19df5b8ac83538fe13fdc37b198cff95c69b59605daa9c" exitCode=0 Jul 24 13:36:47.211889 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.211891 2579 generic.go:356] "Generic (PLEG): container finished" podID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerID="d139208254fbc85d3e9dfc1c233100ed98f80f826465b35e468e77cc97b9db66" exitCode=0 Jul 24 13:36:47.211889 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.211897 2579 generic.go:356] "Generic (PLEG): container finished" podID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerID="073215642ba4662fb18080db20a4009c173356a800fc0da8c4351082a7d00167" exitCode=0 Jul 24 13:36:47.212180 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.211922 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955bb86f-1d3e-4ae1-9b25-8725023194a4","Type":"ContainerDied","Data":"8d24d5147f1e712f545910a545b7f2a94dee6048921f821ea2152050b5d4d367"} Jul 24 13:36:47.212180 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.211985 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955bb86f-1d3e-4ae1-9b25-8725023194a4","Type":"ContainerDied","Data":"89d0bc92dffa8ed09a19df5b8ac83538fe13fdc37b198cff95c69b59605daa9c"} Jul 24 13:36:47.212180 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.212028 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955bb86f-1d3e-4ae1-9b25-8725023194a4","Type":"ContainerDied","Data":"d139208254fbc85d3e9dfc1c233100ed98f80f826465b35e468e77cc97b9db66"} Jul 24 13:36:47.212180 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.212054 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955bb86f-1d3e-4ae1-9b25-8725023194a4","Type":"ContainerDied","Data":"073215642ba4662fb18080db20a4009c173356a800fc0da8c4351082a7d00167"} Jul 24 13:36:47.637726 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.637705 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:47.733748 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.733662 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/955bb86f-1d3e-4ae1-9b25-8725023194a4-alertmanager-main-db\") pod \"955bb86f-1d3e-4ae1-9b25-8725023194a4\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " Jul 24 13:36:47.733748 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.733706 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxq58\" (UniqueName: \"kubernetes.io/projected/955bb86f-1d3e-4ae1-9b25-8725023194a4-kube-api-access-kxq58\") pod \"955bb86f-1d3e-4ae1-9b25-8725023194a4\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " Jul 24 13:36:47.733748 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.733734 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-config-volume\") pod \"955bb86f-1d3e-4ae1-9b25-8725023194a4\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " Jul 24 13:36:47.734047 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.733756 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/955bb86f-1d3e-4ae1-9b25-8725023194a4-alertmanager-trusted-ca-bundle\") pod \"955bb86f-1d3e-4ae1-9b25-8725023194a4\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " Jul 24 13:36:47.734047 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.733783 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/955bb86f-1d3e-4ae1-9b25-8725023194a4-config-out\") pod \"955bb86f-1d3e-4ae1-9b25-8725023194a4\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " Jul 24 13:36:47.734047 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.733812 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-kube-rbac-proxy-web\") pod \"955bb86f-1d3e-4ae1-9b25-8725023194a4\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " Jul 24 13:36:47.734047 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.733840 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-kube-rbac-proxy\") pod \"955bb86f-1d3e-4ae1-9b25-8725023194a4\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " Jul 24 13:36:47.734047 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.733874 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-web-config\") pod \"955bb86f-1d3e-4ae1-9b25-8725023194a4\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " Jul 24 13:36:47.734047 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.733904 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/955bb86f-1d3e-4ae1-9b25-8725023194a4-tls-assets\") pod \"955bb86f-1d3e-4ae1-9b25-8725023194a4\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " Jul 24 13:36:47.734047 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.733950 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"955bb86f-1d3e-4ae1-9b25-8725023194a4\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " Jul 24 13:36:47.734047 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.734024 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/955bb86f-1d3e-4ae1-9b25-8725023194a4-metrics-client-ca\") pod \"955bb86f-1d3e-4ae1-9b25-8725023194a4\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " Jul 24 13:36:47.734423 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.734055 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-main-tls\") pod \"955bb86f-1d3e-4ae1-9b25-8725023194a4\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " Jul 24 13:36:47.734423 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.734084 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-cluster-tls-config\") pod \"955bb86f-1d3e-4ae1-9b25-8725023194a4\" (UID: \"955bb86f-1d3e-4ae1-9b25-8725023194a4\") " Jul 24 13:36:47.734423 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.734076 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/955bb86f-1d3e-4ae1-9b25-8725023194a4-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "955bb86f-1d3e-4ae1-9b25-8725023194a4" (UID: "955bb86f-1d3e-4ae1-9b25-8725023194a4"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:36:47.734423 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.734235 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/955bb86f-1d3e-4ae1-9b25-8725023194a4-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "955bb86f-1d3e-4ae1-9b25-8725023194a4" (UID: "955bb86f-1d3e-4ae1-9b25-8725023194a4"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:36:47.734423 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.734364 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/955bb86f-1d3e-4ae1-9b25-8725023194a4-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "955bb86f-1d3e-4ae1-9b25-8725023194a4" (UID: "955bb86f-1d3e-4ae1-9b25-8725023194a4"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:36:47.737230 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.737185 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/955bb86f-1d3e-4ae1-9b25-8725023194a4-kube-api-access-kxq58" (OuterVolumeSpecName: "kube-api-access-kxq58") pod "955bb86f-1d3e-4ae1-9b25-8725023194a4" (UID: "955bb86f-1d3e-4ae1-9b25-8725023194a4"). InnerVolumeSpecName "kube-api-access-kxq58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:36:47.737380 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.737308 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/955bb86f-1d3e-4ae1-9b25-8725023194a4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "955bb86f-1d3e-4ae1-9b25-8725023194a4" (UID: "955bb86f-1d3e-4ae1-9b25-8725023194a4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:36:47.737441 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.737406 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "955bb86f-1d3e-4ae1-9b25-8725023194a4" (UID: "955bb86f-1d3e-4ae1-9b25-8725023194a4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:36:47.737802 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.737750 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-config-volume" (OuterVolumeSpecName: "config-volume") pod "955bb86f-1d3e-4ae1-9b25-8725023194a4" (UID: "955bb86f-1d3e-4ae1-9b25-8725023194a4"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:36:47.737915 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.737824 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/955bb86f-1d3e-4ae1-9b25-8725023194a4-config-out" (OuterVolumeSpecName: "config-out") pod "955bb86f-1d3e-4ae1-9b25-8725023194a4" (UID: "955bb86f-1d3e-4ae1-9b25-8725023194a4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:36:47.737915 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.737846 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "955bb86f-1d3e-4ae1-9b25-8725023194a4" (UID: "955bb86f-1d3e-4ae1-9b25-8725023194a4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:36:47.738048 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.737925 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "955bb86f-1d3e-4ae1-9b25-8725023194a4" (UID: "955bb86f-1d3e-4ae1-9b25-8725023194a4"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:36:47.738048 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.738012 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "955bb86f-1d3e-4ae1-9b25-8725023194a4" (UID: "955bb86f-1d3e-4ae1-9b25-8725023194a4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:36:47.742525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.742503 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "955bb86f-1d3e-4ae1-9b25-8725023194a4" (UID: "955bb86f-1d3e-4ae1-9b25-8725023194a4"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:36:47.748480 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.748458 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-web-config" (OuterVolumeSpecName: "web-config") pod "955bb86f-1d3e-4ae1-9b25-8725023194a4" (UID: "955bb86f-1d3e-4ae1-9b25-8725023194a4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:36:47.835650 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.835607 2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/955bb86f-1d3e-4ae1-9b25-8725023194a4-tls-assets\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:36:47.835650 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.835645 2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:36:47.835650 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.835657 2579 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/955bb86f-1d3e-4ae1-9b25-8725023194a4-metrics-client-ca\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:36:47.835650 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.835668 2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-main-tls\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:36:47.835915 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.835677 2579 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-cluster-tls-config\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:36:47.835915 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.835685 2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/955bb86f-1d3e-4ae1-9b25-8725023194a4-alertmanager-main-db\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:36:47.835915 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.835693 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxq58\" (UniqueName: \"kubernetes.io/projected/955bb86f-1d3e-4ae1-9b25-8725023194a4-kube-api-access-kxq58\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:36:47.835915 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.835701 2579 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-config-volume\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:36:47.835915 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.835710 2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/955bb86f-1d3e-4ae1-9b25-8725023194a4-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:36:47.835915 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.835718 2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/955bb86f-1d3e-4ae1-9b25-8725023194a4-config-out\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:36:47.835915 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.835726 2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:36:47.835915 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.835734 2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:36:47.835915 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:47.835743 2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/955bb86f-1d3e-4ae1-9b25-8725023194a4-web-config\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:36:48.218515 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.218479 2579 generic.go:356] "Generic (PLEG): container finished" podID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerID="c2bdf763884ced2f7c02b0440247ffcd71b0a1e505ae6d16f2d0f676fdbe067f" exitCode=0 Jul 24 13:36:48.218515 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.218512 2579 generic.go:356] "Generic (PLEG): container finished" podID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerID="8596d554b27af0a1b4c4e24853e9851ffd9d9cf2fc6fbdb630d0312b9c78e45c" exitCode=0 Jul 24 13:36:48.218726 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.218572 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.218726 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.218573 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955bb86f-1d3e-4ae1-9b25-8725023194a4","Type":"ContainerDied","Data":"c2bdf763884ced2f7c02b0440247ffcd71b0a1e505ae6d16f2d0f676fdbe067f"} Jul 24 13:36:48.218726 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.218642 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955bb86f-1d3e-4ae1-9b25-8725023194a4","Type":"ContainerDied","Data":"8596d554b27af0a1b4c4e24853e9851ffd9d9cf2fc6fbdb630d0312b9c78e45c"} Jul 24 13:36:48.218726 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.218673 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955bb86f-1d3e-4ae1-9b25-8725023194a4","Type":"ContainerDied","Data":"494312f9a10c6ed03273402576ac835af04bd9b7ad3fd8f6350c71082bf7e9ef"} Jul 24 13:36:48.218726 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.218704 2579 scope.go:117] "RemoveContainer" containerID="8d24d5147f1e712f545910a545b7f2a94dee6048921f821ea2152050b5d4d367" Jul 24 13:36:48.232585 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.232564 2579 scope.go:117] "RemoveContainer" containerID="c2bdf763884ced2f7c02b0440247ffcd71b0a1e505ae6d16f2d0f676fdbe067f" Jul 24 13:36:48.239670 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.239652 2579 scope.go:117] "RemoveContainer" containerID="89d0bc92dffa8ed09a19df5b8ac83538fe13fdc37b198cff95c69b59605daa9c" Jul 24 13:36:48.246180 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.246161 2579 scope.go:117] "RemoveContainer" containerID="8596d554b27af0a1b4c4e24853e9851ffd9d9cf2fc6fbdb630d0312b9c78e45c" Jul 24 13:36:48.252450 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.252430 2579 scope.go:117] "RemoveContainer" containerID="d139208254fbc85d3e9dfc1c233100ed98f80f826465b35e468e77cc97b9db66" Jul 24 13:36:48.259724 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.259701 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 13:36:48.260026 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.260009 2579 scope.go:117] "RemoveContainer" containerID="073215642ba4662fb18080db20a4009c173356a800fc0da8c4351082a7d00167" Jul 24 13:36:48.267172 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.267087 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 13:36:48.267225 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.267182 2579 scope.go:117] "RemoveContainer" containerID="bee6d482dff4bc989747b904bb435bd85c860ca3bf3c70a0d63a135f13f06398" Jul 24 13:36:48.273952 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.273933 2579 scope.go:117] "RemoveContainer" containerID="8d24d5147f1e712f545910a545b7f2a94dee6048921f821ea2152050b5d4d367" Jul 24 13:36:48.274237 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:36:48.274217 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d24d5147f1e712f545910a545b7f2a94dee6048921f821ea2152050b5d4d367\": container with ID starting with 8d24d5147f1e712f545910a545b7f2a94dee6048921f821ea2152050b5d4d367 not found: ID does not exist" containerID="8d24d5147f1e712f545910a545b7f2a94dee6048921f821ea2152050b5d4d367" Jul 24 13:36:48.274302 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.274246 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d24d5147f1e712f545910a545b7f2a94dee6048921f821ea2152050b5d4d367"} err="failed to get container status \"8d24d5147f1e712f545910a545b7f2a94dee6048921f821ea2152050b5d4d367\": rpc error: code = NotFound desc = could not find container \"8d24d5147f1e712f545910a545b7f2a94dee6048921f821ea2152050b5d4d367\": container with ID starting with 8d24d5147f1e712f545910a545b7f2a94dee6048921f821ea2152050b5d4d367 not found: ID does not exist" Jul 24 13:36:48.274302 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.274278 2579 scope.go:117] "RemoveContainer" containerID="c2bdf763884ced2f7c02b0440247ffcd71b0a1e505ae6d16f2d0f676fdbe067f" Jul 24 13:36:48.274508 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:36:48.274493 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2bdf763884ced2f7c02b0440247ffcd71b0a1e505ae6d16f2d0f676fdbe067f\": container with ID starting with c2bdf763884ced2f7c02b0440247ffcd71b0a1e505ae6d16f2d0f676fdbe067f not found: ID does not exist" containerID="c2bdf763884ced2f7c02b0440247ffcd71b0a1e505ae6d16f2d0f676fdbe067f" Jul 24 13:36:48.274548 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.274512 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2bdf763884ced2f7c02b0440247ffcd71b0a1e505ae6d16f2d0f676fdbe067f"} err="failed to get container status \"c2bdf763884ced2f7c02b0440247ffcd71b0a1e505ae6d16f2d0f676fdbe067f\": rpc error: code = NotFound desc = could not find container \"c2bdf763884ced2f7c02b0440247ffcd71b0a1e505ae6d16f2d0f676fdbe067f\": container with ID starting with c2bdf763884ced2f7c02b0440247ffcd71b0a1e505ae6d16f2d0f676fdbe067f not found: ID does not exist" Jul 24 13:36:48.274548 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.274523 2579 scope.go:117] "RemoveContainer" containerID="89d0bc92dffa8ed09a19df5b8ac83538fe13fdc37b198cff95c69b59605daa9c" Jul 24 13:36:48.274715 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:36:48.274700 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89d0bc92dffa8ed09a19df5b8ac83538fe13fdc37b198cff95c69b59605daa9c\": container with ID starting with 89d0bc92dffa8ed09a19df5b8ac83538fe13fdc37b198cff95c69b59605daa9c not found: ID does not exist" containerID="89d0bc92dffa8ed09a19df5b8ac83538fe13fdc37b198cff95c69b59605daa9c" Jul 24 13:36:48.274757 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.274719 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89d0bc92dffa8ed09a19df5b8ac83538fe13fdc37b198cff95c69b59605daa9c"} err="failed to get container status \"89d0bc92dffa8ed09a19df5b8ac83538fe13fdc37b198cff95c69b59605daa9c\": rpc error: code = NotFound desc = could not find container \"89d0bc92dffa8ed09a19df5b8ac83538fe13fdc37b198cff95c69b59605daa9c\": container with ID starting with 89d0bc92dffa8ed09a19df5b8ac83538fe13fdc37b198cff95c69b59605daa9c not found: ID does not exist" Jul 24 13:36:48.274757 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.274731 2579 scope.go:117] "RemoveContainer" containerID="8596d554b27af0a1b4c4e24853e9851ffd9d9cf2fc6fbdb630d0312b9c78e45c" Jul 24 13:36:48.274917 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:36:48.274902 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8596d554b27af0a1b4c4e24853e9851ffd9d9cf2fc6fbdb630d0312b9c78e45c\": container with ID starting with 8596d554b27af0a1b4c4e24853e9851ffd9d9cf2fc6fbdb630d0312b9c78e45c not found: ID does not exist" containerID="8596d554b27af0a1b4c4e24853e9851ffd9d9cf2fc6fbdb630d0312b9c78e45c" Jul 24 13:36:48.274958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.274919 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8596d554b27af0a1b4c4e24853e9851ffd9d9cf2fc6fbdb630d0312b9c78e45c"} err="failed to get container status \"8596d554b27af0a1b4c4e24853e9851ffd9d9cf2fc6fbdb630d0312b9c78e45c\": rpc error: code = NotFound desc = could not find container \"8596d554b27af0a1b4c4e24853e9851ffd9d9cf2fc6fbdb630d0312b9c78e45c\": container with ID starting with 8596d554b27af0a1b4c4e24853e9851ffd9d9cf2fc6fbdb630d0312b9c78e45c not found: ID does not exist" Jul 24 13:36:48.274958 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.274930 2579 scope.go:117] "RemoveContainer" containerID="d139208254fbc85d3e9dfc1c233100ed98f80f826465b35e468e77cc97b9db66" Jul 24 13:36:48.275181 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:36:48.275164 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d139208254fbc85d3e9dfc1c233100ed98f80f826465b35e468e77cc97b9db66\": container with ID starting with d139208254fbc85d3e9dfc1c233100ed98f80f826465b35e468e77cc97b9db66 not found: ID does not exist" containerID="d139208254fbc85d3e9dfc1c233100ed98f80f826465b35e468e77cc97b9db66" Jul 24 13:36:48.275226 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.275186 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d139208254fbc85d3e9dfc1c233100ed98f80f826465b35e468e77cc97b9db66"} err="failed to get container status \"d139208254fbc85d3e9dfc1c233100ed98f80f826465b35e468e77cc97b9db66\": rpc error: code = NotFound desc = could not find container \"d139208254fbc85d3e9dfc1c233100ed98f80f826465b35e468e77cc97b9db66\": container with ID starting with d139208254fbc85d3e9dfc1c233100ed98f80f826465b35e468e77cc97b9db66 not found: ID does not exist" Jul 24 13:36:48.275226 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.275202 2579 scope.go:117] "RemoveContainer" containerID="073215642ba4662fb18080db20a4009c173356a800fc0da8c4351082a7d00167" Jul 24 13:36:48.275429 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:36:48.275412 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"073215642ba4662fb18080db20a4009c173356a800fc0da8c4351082a7d00167\": container with ID starting with 073215642ba4662fb18080db20a4009c173356a800fc0da8c4351082a7d00167 not found: ID does not exist" containerID="073215642ba4662fb18080db20a4009c173356a800fc0da8c4351082a7d00167" Jul 24 13:36:48.275473 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.275433 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"073215642ba4662fb18080db20a4009c173356a800fc0da8c4351082a7d00167"} err="failed to get container status \"073215642ba4662fb18080db20a4009c173356a800fc0da8c4351082a7d00167\": rpc error: code = NotFound desc = could not find container \"073215642ba4662fb18080db20a4009c173356a800fc0da8c4351082a7d00167\": container with ID starting with 073215642ba4662fb18080db20a4009c173356a800fc0da8c4351082a7d00167 not found: ID does not exist" Jul 24 13:36:48.275473 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.275446 2579 scope.go:117] "RemoveContainer" containerID="bee6d482dff4bc989747b904bb435bd85c860ca3bf3c70a0d63a135f13f06398" Jul 24 13:36:48.275684 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:36:48.275667 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bee6d482dff4bc989747b904bb435bd85c860ca3bf3c70a0d63a135f13f06398\": container with ID starting with bee6d482dff4bc989747b904bb435bd85c860ca3bf3c70a0d63a135f13f06398 not found: ID does not exist" containerID="bee6d482dff4bc989747b904bb435bd85c860ca3bf3c70a0d63a135f13f06398" Jul 24 13:36:48.275725 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.275690 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bee6d482dff4bc989747b904bb435bd85c860ca3bf3c70a0d63a135f13f06398"} err="failed to get container status \"bee6d482dff4bc989747b904bb435bd85c860ca3bf3c70a0d63a135f13f06398\": rpc error: code = NotFound desc = could not find container \"bee6d482dff4bc989747b904bb435bd85c860ca3bf3c70a0d63a135f13f06398\": container with ID starting with bee6d482dff4bc989747b904bb435bd85c860ca3bf3c70a0d63a135f13f06398 not found: ID does not exist" Jul 24 13:36:48.275725 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.275706 2579 scope.go:117] "RemoveContainer" containerID="8d24d5147f1e712f545910a545b7f2a94dee6048921f821ea2152050b5d4d367" Jul 24 13:36:48.275907 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.275891 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d24d5147f1e712f545910a545b7f2a94dee6048921f821ea2152050b5d4d367"} err="failed to get container status \"8d24d5147f1e712f545910a545b7f2a94dee6048921f821ea2152050b5d4d367\": rpc error: code = NotFound desc = could not find container \"8d24d5147f1e712f545910a545b7f2a94dee6048921f821ea2152050b5d4d367\": container with ID starting with 8d24d5147f1e712f545910a545b7f2a94dee6048921f821ea2152050b5d4d367 not found: ID does not exist" Jul 24 13:36:48.275946 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.275907 2579 scope.go:117] "RemoveContainer" containerID="c2bdf763884ced2f7c02b0440247ffcd71b0a1e505ae6d16f2d0f676fdbe067f" Jul 24 13:36:48.276129 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.276112 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2bdf763884ced2f7c02b0440247ffcd71b0a1e505ae6d16f2d0f676fdbe067f"} err="failed to get container status \"c2bdf763884ced2f7c02b0440247ffcd71b0a1e505ae6d16f2d0f676fdbe067f\": rpc error: code = NotFound desc = could not find container \"c2bdf763884ced2f7c02b0440247ffcd71b0a1e505ae6d16f2d0f676fdbe067f\": container with ID starting with c2bdf763884ced2f7c02b0440247ffcd71b0a1e505ae6d16f2d0f676fdbe067f not found: ID does not exist" Jul 24 13:36:48.276186 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.276130 2579 scope.go:117] "RemoveContainer" containerID="89d0bc92dffa8ed09a19df5b8ac83538fe13fdc37b198cff95c69b59605daa9c" Jul 24 13:36:48.276322 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.276296 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89d0bc92dffa8ed09a19df5b8ac83538fe13fdc37b198cff95c69b59605daa9c"} err="failed to get container status \"89d0bc92dffa8ed09a19df5b8ac83538fe13fdc37b198cff95c69b59605daa9c\": rpc error: code = NotFound desc = could not find container \"89d0bc92dffa8ed09a19df5b8ac83538fe13fdc37b198cff95c69b59605daa9c\": container with ID starting with 89d0bc92dffa8ed09a19df5b8ac83538fe13fdc37b198cff95c69b59605daa9c not found: ID does not exist" Jul 24 13:36:48.276365 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.276323 2579 scope.go:117] "RemoveContainer" containerID="8596d554b27af0a1b4c4e24853e9851ffd9d9cf2fc6fbdb630d0312b9c78e45c" Jul 24 13:36:48.276555 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.276537 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8596d554b27af0a1b4c4e24853e9851ffd9d9cf2fc6fbdb630d0312b9c78e45c"} err="failed to get container status \"8596d554b27af0a1b4c4e24853e9851ffd9d9cf2fc6fbdb630d0312b9c78e45c\": rpc error: code = NotFound desc = could not find container \"8596d554b27af0a1b4c4e24853e9851ffd9d9cf2fc6fbdb630d0312b9c78e45c\": container with ID starting with 8596d554b27af0a1b4c4e24853e9851ffd9d9cf2fc6fbdb630d0312b9c78e45c not found: ID does not exist" Jul 24 13:36:48.276601 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.276555 2579 scope.go:117] "RemoveContainer" containerID="d139208254fbc85d3e9dfc1c233100ed98f80f826465b35e468e77cc97b9db66" Jul 24 13:36:48.276783 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.276766 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d139208254fbc85d3e9dfc1c233100ed98f80f826465b35e468e77cc97b9db66"} err="failed to get container status \"d139208254fbc85d3e9dfc1c233100ed98f80f826465b35e468e77cc97b9db66\": rpc error: code = NotFound desc = could not find container \"d139208254fbc85d3e9dfc1c233100ed98f80f826465b35e468e77cc97b9db66\": container with ID starting with d139208254fbc85d3e9dfc1c233100ed98f80f826465b35e468e77cc97b9db66 not found: ID does not exist" Jul 24 13:36:48.276827 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.276783 2579 scope.go:117] "RemoveContainer" containerID="073215642ba4662fb18080db20a4009c173356a800fc0da8c4351082a7d00167" Jul 24 13:36:48.276980 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.276964 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"073215642ba4662fb18080db20a4009c173356a800fc0da8c4351082a7d00167"} err="failed to get container status \"073215642ba4662fb18080db20a4009c173356a800fc0da8c4351082a7d00167\": rpc error: code = NotFound desc = could not find container \"073215642ba4662fb18080db20a4009c173356a800fc0da8c4351082a7d00167\": container with ID starting with 073215642ba4662fb18080db20a4009c173356a800fc0da8c4351082a7d00167 not found: ID does not exist" Jul 24 13:36:48.277051 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.276981 2579 scope.go:117] "RemoveContainer" containerID="bee6d482dff4bc989747b904bb435bd85c860ca3bf3c70a0d63a135f13f06398" Jul 24 13:36:48.277211 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.277196 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bee6d482dff4bc989747b904bb435bd85c860ca3bf3c70a0d63a135f13f06398"} err="failed to get container status \"bee6d482dff4bc989747b904bb435bd85c860ca3bf3c70a0d63a135f13f06398\": rpc error: code = NotFound desc = could not find container \"bee6d482dff4bc989747b904bb435bd85c860ca3bf3c70a0d63a135f13f06398\": container with ID starting with bee6d482dff4bc989747b904bb435bd85c860ca3bf3c70a0d63a135f13f06398 not found: ID does not exist" Jul 24 13:36:48.306395 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306362 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 13:36:48.306629 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306618 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="prom-label-proxy" Jul 24 13:36:48.306674 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306632 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="prom-label-proxy" Jul 24 13:36:48.306674 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306642 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="config-reloader" Jul 24 13:36:48.306674 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306647 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="config-reloader" Jul 24 13:36:48.306674 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306654 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="kube-rbac-proxy" Jul 24 13:36:48.306674 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306659 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="kube-rbac-proxy" Jul 24 13:36:48.306674 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306665 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="kube-rbac-proxy-metric" Jul 24 13:36:48.306674 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306670 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="kube-rbac-proxy-metric" Jul 24 13:36:48.306863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306678 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="kube-rbac-proxy-web" Jul 24 13:36:48.306863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306683 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="kube-rbac-proxy-web" Jul 24 13:36:48.306863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306690 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="init-config-reloader" Jul 24 13:36:48.306863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306695 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="init-config-reloader" Jul 24 13:36:48.306863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306705 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="alertmanager" Jul 24 13:36:48.306863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306710 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="alertmanager" Jul 24 13:36:48.306863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306750 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="alertmanager" Jul 24 13:36:48.306863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306757 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="kube-rbac-proxy-web" Jul 24 13:36:48.306863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306765 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="prom-label-proxy" Jul 24 13:36:48.306863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306773 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="config-reloader" Jul 24 13:36:48.306863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306778 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="kube-rbac-proxy-metric" Jul 24 13:36:48.306863 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.306783 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" containerName="kube-rbac-proxy" Jul 24 13:36:48.310479 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.310465 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.314750 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.314729 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 24 13:36:48.314860 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.314815 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 24 13:36:48.314984 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.314971 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 24 13:36:48.315240 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.315225 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 24 13:36:48.315324 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.315254 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tbmt9\"" Jul 24 13:36:48.315324 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.315313 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 24 13:36:48.315626 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.315609 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 24 13:36:48.315705 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.315615 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 24 13:36:48.316127 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.316106 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 24 13:36:48.322204 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.322186 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 24 13:36:48.338349 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.338321 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 13:36:48.417912 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.417869 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="955bb86f-1d3e-4ae1-9b25-8725023194a4" path="/var/lib/kubelet/pods/955bb86f-1d3e-4ae1-9b25-8725023194a4/volumes" Jul 24 13:36:48.440417 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.440384 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d49f3a1a-c07e-4ea0-baac-94778e221398-config-out\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.440417 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.440418 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d49f3a1a-c07e-4ea0-baac-94778e221398-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.440620 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.440439 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.440620 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.440456 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.440620 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.440569 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d49f3a1a-c07e-4ea0-baac-94778e221398-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.440620 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.440599 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.440620 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.440616 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v79nv\" (UniqueName: \"kubernetes.io/projected/d49f3a1a-c07e-4ea0-baac-94778e221398-kube-api-access-v79nv\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.440803 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.440666 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-config-volume\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.440803 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.440682 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-web-config\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.440803 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.440705 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d49f3a1a-c07e-4ea0-baac-94778e221398-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.440803 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.440758 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.440803 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.440773 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.440803 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.440794 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d49f3a1a-c07e-4ea0-baac-94778e221398-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.541560 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.541527 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-config-volume\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.541560 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.541557 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-web-config\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.541742 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.541578 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d49f3a1a-c07e-4ea0-baac-94778e221398-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.541742 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.541615 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.541809 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.541733 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.541809 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.541784 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d49f3a1a-c07e-4ea0-baac-94778e221398-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.542186 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.541829 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d49f3a1a-c07e-4ea0-baac-94778e221398-config-out\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.542186 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.541864 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d49f3a1a-c07e-4ea0-baac-94778e221398-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.542186 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.541888 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.542186 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.541913 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.542186 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.541960 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d49f3a1a-c07e-4ea0-baac-94778e221398-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.542186 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.541977 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d49f3a1a-c07e-4ea0-baac-94778e221398-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.542186 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.542036 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.542186 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.542065 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v79nv\" (UniqueName: \"kubernetes.io/projected/d49f3a1a-c07e-4ea0-baac-94778e221398-kube-api-access-v79nv\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.542655 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.542561 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d49f3a1a-c07e-4ea0-baac-94778e221398-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.545079 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.544885 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-config-volume\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.545079 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.544931 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.545079 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.544934 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.545079 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.545033 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.545352 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.545247 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-web-config\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.545352 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.545340 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.545463 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.545425 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d49f3a1a-c07e-4ea0-baac-94778e221398-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.545832 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.545813 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d49f3a1a-c07e-4ea0-baac-94778e221398-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.545948 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.545935 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d49f3a1a-c07e-4ea0-baac-94778e221398-config-out\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.547332 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.547316 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d49f3a1a-c07e-4ea0-baac-94778e221398-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.555351 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.555333 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v79nv\" (UniqueName: \"kubernetes.io/projected/d49f3a1a-c07e-4ea0-baac-94778e221398-kube-api-access-v79nv\") pod \"alertmanager-main-0\" (UID: \"d49f3a1a-c07e-4ea0-baac-94778e221398\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.619952 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.619921 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:36:48.778620 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:48.778542 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 13:36:48.782926 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:36:48.782898 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd49f3a1a_c07e_4ea0_baac_94778e221398.slice/crio-59a94124a391d385585a4f2d1525fba8610191ffda0ed248a4eaf6898366183a WatchSource:0}: Error finding container 59a94124a391d385585a4f2d1525fba8610191ffda0ed248a4eaf6898366183a: Status 404 returned error can't find the container with id 59a94124a391d385585a4f2d1525fba8610191ffda0ed248a4eaf6898366183a Jul 24 13:36:49.223304 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:49.223264 2579 generic.go:356] "Generic (PLEG): container finished" podID="d49f3a1a-c07e-4ea0-baac-94778e221398" containerID="534f4e36fa18ffacdc108754ad19c44a99896a7268fddf4d8534e74e56e5a538" exitCode=0 Jul 24 13:36:49.223502 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:49.223358 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d49f3a1a-c07e-4ea0-baac-94778e221398","Type":"ContainerDied","Data":"534f4e36fa18ffacdc108754ad19c44a99896a7268fddf4d8534e74e56e5a538"} Jul 24 13:36:49.223502 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:49.223437 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d49f3a1a-c07e-4ea0-baac-94778e221398","Type":"ContainerStarted","Data":"59a94124a391d385585a4f2d1525fba8610191ffda0ed248a4eaf6898366183a"} Jul 24 13:36:50.231347 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.231317 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d49f3a1a-c07e-4ea0-baac-94778e221398","Type":"ContainerStarted","Data":"f089f07056fc6c7f9c855ed6fc91ed7dd8ae2a4a49e520fecdbd45e6ad6c7642"} Jul 24 13:36:50.231721 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.231411 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d49f3a1a-c07e-4ea0-baac-94778e221398","Type":"ContainerStarted","Data":"116bcca0aba2120426362b79dbbe049bd7745ec854703f64e6eef2495df30710"} Jul 24 13:36:50.231721 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.231437 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d49f3a1a-c07e-4ea0-baac-94778e221398","Type":"ContainerStarted","Data":"1dab0e9e08caf7767f582c153982fcb7ff34a0551958fc9e2c460641512acbd8"} Jul 24 13:36:50.231721 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.231465 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d49f3a1a-c07e-4ea0-baac-94778e221398","Type":"ContainerStarted","Data":"81ed15d40629f2ae95bdf9609ce85499f700d4c33d779c135ef4588638f665f1"} Jul 24 13:36:50.231721 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.231490 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d49f3a1a-c07e-4ea0-baac-94778e221398","Type":"ContainerStarted","Data":"d6fb185e1ece1756bacd09cbdfe0a495b983a3d9c083014c02f86dcd4a6027bf"} Jul 24 13:36:50.231721 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.231514 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d49f3a1a-c07e-4ea0-baac-94778e221398","Type":"ContainerStarted","Data":"a44e52c619f1980726ffd0c290c9af495512fa72bc4ed516481bebfe67a622ee"} Jul 24 13:36:50.270499 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.270433 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.255495742 podStartE2EDuration="2.27041456s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.918818ms" isStatefulPod=false podCreationTimestamp="2026-07-24 13:36:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:36:50.268076849 +0000 UTC m=+152.391459250" watchObservedRunningTime="2026-07-24 13:36:50.27041456 +0000 UTC m=+152.393796965" Jul 24 13:36:50.440443 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.440408 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv"] Jul 24 13:36:50.443138 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.443114 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.446901 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.446878 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 24 13:36:50.447091 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.447074 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 24 13:36:50.447177 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.447149 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-95k76\"" Jul 24 13:36:50.447177 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.447155 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 24 13:36:50.447177 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.447155 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 24 13:36:50.447509 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.447493 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 24 13:36:50.454723 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.454702 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 24 13:36:50.468259 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.468227 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv"] Jul 24 13:36:50.558995 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.558966 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4d254863-fa16-463a-8f54-7777a6df5d8a-telemeter-client-tls\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.559146 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.559058 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4d254863-fa16-463a-8f54-7777a6df5d8a-federate-client-tls\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.559146 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.559080 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d254863-fa16-463a-8f54-7777a6df5d8a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.559146 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.559103 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lng2\" (UniqueName: \"kubernetes.io/projected/4d254863-fa16-463a-8f54-7777a6df5d8a-kube-api-access-8lng2\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.559146 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.559135 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d254863-fa16-463a-8f54-7777a6df5d8a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.559306 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.559228 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d254863-fa16-463a-8f54-7777a6df5d8a-serving-certs-ca-bundle\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.559306 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.559293 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4d254863-fa16-463a-8f54-7777a6df5d8a-secret-telemeter-client\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.559410 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.559322 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d254863-fa16-463a-8f54-7777a6df5d8a-metrics-client-ca\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.660456 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.660417 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d254863-fa16-463a-8f54-7777a6df5d8a-serving-certs-ca-bundle\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.660650 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.660474 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4d254863-fa16-463a-8f54-7777a6df5d8a-secret-telemeter-client\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.660650 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.660500 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d254863-fa16-463a-8f54-7777a6df5d8a-metrics-client-ca\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.660650 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.660522 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4d254863-fa16-463a-8f54-7777a6df5d8a-telemeter-client-tls\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.660650 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.660554 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4d254863-fa16-463a-8f54-7777a6df5d8a-federate-client-tls\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.660650 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.660575 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d254863-fa16-463a-8f54-7777a6df5d8a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.660650 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.660604 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lng2\" (UniqueName: \"kubernetes.io/projected/4d254863-fa16-463a-8f54-7777a6df5d8a-kube-api-access-8lng2\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.660650 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.660627 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d254863-fa16-463a-8f54-7777a6df5d8a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.661315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.661271 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d254863-fa16-463a-8f54-7777a6df5d8a-serving-certs-ca-bundle\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.661315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.661284 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d254863-fa16-463a-8f54-7777a6df5d8a-metrics-client-ca\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.661471 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.661447 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d254863-fa16-463a-8f54-7777a6df5d8a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.663230 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.663210 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4d254863-fa16-463a-8f54-7777a6df5d8a-federate-client-tls\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.663328 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.663303 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d254863-fa16-463a-8f54-7777a6df5d8a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.663610 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.663595 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4d254863-fa16-463a-8f54-7777a6df5d8a-telemeter-client-tls\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.663675 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.663657 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4d254863-fa16-463a-8f54-7777a6df5d8a-secret-telemeter-client\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.671279 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.671260 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lng2\" (UniqueName: \"kubernetes.io/projected/4d254863-fa16-463a-8f54-7777a6df5d8a-kube-api-access-8lng2\") pod \"telemeter-client-7c7b48dcb7-pwhvv\" (UID: \"4d254863-fa16-463a-8f54-7777a6df5d8a\") " pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.751918 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.751886 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" Jul 24 13:36:50.902538 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:50.902508 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv"] Jul 24 13:36:50.904826 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:36:50.904799 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d254863_fa16_463a_8f54_7777a6df5d8a.slice/crio-e5ff56f8b11e44b8573a46a903a52dca28aa808a01a2239110fcaaf2fd96a0a3 WatchSource:0}: Error finding container e5ff56f8b11e44b8573a46a903a52dca28aa808a01a2239110fcaaf2fd96a0a3: Status 404 returned error can't find the container with id e5ff56f8b11e44b8573a46a903a52dca28aa808a01a2239110fcaaf2fd96a0a3 Jul 24 13:36:51.239388 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:51.239297 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" event={"ID":"4d254863-fa16-463a-8f54-7777a6df5d8a","Type":"ContainerStarted","Data":"e5ff56f8b11e44b8573a46a903a52dca28aa808a01a2239110fcaaf2fd96a0a3"} Jul 24 13:36:53.249986 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:53.249948 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" event={"ID":"4d254863-fa16-463a-8f54-7777a6df5d8a","Type":"ContainerStarted","Data":"959072f954e9f137a7fcb718c520eed97dab7d9fe6c0e66c5a12f1ef94b005b6"} Jul 24 13:36:53.250376 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:53.250044 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" event={"ID":"4d254863-fa16-463a-8f54-7777a6df5d8a","Type":"ContainerStarted","Data":"4073bd88d4163546f250e4ca4963c29b3d1d5071d3805a47a8a315ab26255c05"} Jul 24 13:36:53.250376 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:53.250075 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" event={"ID":"4d254863-fa16-463a-8f54-7777a6df5d8a","Type":"ContainerStarted","Data":"a614455f6afc2e29ed62b227724c6de8b11768d568f5ea5d9a883945b9f2c94e"} Jul 24 13:36:53.282945 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:53.282895 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7c7b48dcb7-pwhvv" podStartSLOduration=1.5462956220000001 podStartE2EDuration="3.282880727s" totalImagesPullingTime="1.736585105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:36:53.280695137 +0000 UTC m=+155.404077534" watchObservedRunningTime="2026-07-24 13:36:53.282880727 +0000 UTC m=+155.406263125" Jul 24 13:36:54.230575 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:54.230541 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:54.322810 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:54.322773 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:59.892790 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:36:59.892734 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:05.220354 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:37:05.220307 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:05.321953 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:37:05.321915 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:05.414000 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:37:05.413963 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:12.287777 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:37:12.287695 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:12.498116 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:37:12.498072 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:28.281343 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:37:28.281306 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:28.365278 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:37:28.365234 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:29.853500 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:37:29.853467 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:00.748075 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:38:00.748035 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:00.838120 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:38:00.838085 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:04.457626 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:38:04.457585 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:04.532323 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:38:04.532269 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:12.670494 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:38:12.670458 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:12.740301 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:38:12.740265 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:15.242965 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:38:15.242924 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:15.342970 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:38:15.342935 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:15.485610 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:38:15.485576 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:15.566269 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:38:15.566235 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:47.630434 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:38:47.630352 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:58.309534 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:38:58.309478 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:58.408482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:38:58.408445 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:02.780980 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:02.780938 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb"] Jul 24 13:39:02.783459 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:02.783439 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb" Jul 24 13:39:02.786806 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:02.786786 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-t7rb5\"" Jul 24 13:39:02.787017 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:02.787006 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 13:39:02.788075 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:02.788060 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 13:39:02.796672 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:02.796644 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb"] Jul 24 13:39:02.835319 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:02.835286 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c84771f-0639-4895-aa2e-8004a4b5df08-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb\" (UID: \"6c84771f-0639-4895-aa2e-8004a4b5df08\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb" Jul 24 13:39:02.835479 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:02.835334 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92bth\" (UniqueName: \"kubernetes.io/projected/6c84771f-0639-4895-aa2e-8004a4b5df08-kube-api-access-92bth\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb\" (UID: \"6c84771f-0639-4895-aa2e-8004a4b5df08\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb" Jul 24 13:39:02.835479 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:02.835401 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c84771f-0639-4895-aa2e-8004a4b5df08-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb\" (UID: \"6c84771f-0639-4895-aa2e-8004a4b5df08\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb" Jul 24 13:39:02.936606 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:02.936571 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c84771f-0639-4895-aa2e-8004a4b5df08-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb\" (UID: \"6c84771f-0639-4895-aa2e-8004a4b5df08\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb" Jul 24 13:39:02.936774 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:02.936634 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92bth\" (UniqueName: \"kubernetes.io/projected/6c84771f-0639-4895-aa2e-8004a4b5df08-kube-api-access-92bth\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb\" (UID: \"6c84771f-0639-4895-aa2e-8004a4b5df08\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb" Jul 24 13:39:02.936774 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:02.936673 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c84771f-0639-4895-aa2e-8004a4b5df08-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb\" (UID: \"6c84771f-0639-4895-aa2e-8004a4b5df08\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb" Jul 24 13:39:02.937103 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:02.937080 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c84771f-0639-4895-aa2e-8004a4b5df08-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb\" (UID: \"6c84771f-0639-4895-aa2e-8004a4b5df08\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb" Jul 24 13:39:02.937154 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:02.937087 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c84771f-0639-4895-aa2e-8004a4b5df08-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb\" (UID: \"6c84771f-0639-4895-aa2e-8004a4b5df08\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb" Jul 24 13:39:02.946190 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:02.946163 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92bth\" (UniqueName: \"kubernetes.io/projected/6c84771f-0639-4895-aa2e-8004a4b5df08-kube-api-access-92bth\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb\" (UID: \"6c84771f-0639-4895-aa2e-8004a4b5df08\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb" Jul 24 13:39:03.092696 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:03.092663 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb" Jul 24 13:39:03.227568 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:03.227534 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb"] Jul 24 13:39:03.230660 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:39:03.230636 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c84771f_0639_4895_aa2e_8004a4b5df08.slice/crio-8aeb3d9369918a75fa337ce325c157ec555e0320e76897d6ff01f63ef5022941 WatchSource:0}: Error finding container 8aeb3d9369918a75fa337ce325c157ec555e0320e76897d6ff01f63ef5022941: Status 404 returned error can't find the container with id 8aeb3d9369918a75fa337ce325c157ec555e0320e76897d6ff01f63ef5022941 Jul 24 13:39:03.709720 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:03.709689 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb" event={"ID":"6c84771f-0639-4895-aa2e-8004a4b5df08","Type":"ContainerStarted","Data":"8aeb3d9369918a75fa337ce325c157ec555e0320e76897d6ff01f63ef5022941"} Jul 24 13:39:09.737681 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:09.737597 2579 generic.go:356] "Generic (PLEG): container finished" podID="6c84771f-0639-4895-aa2e-8004a4b5df08" containerID="a9d2a03b636c4fa4ce8e26bdfe6640cc2d1e8522ce0bb98ca6d9e9c87212c0b0" exitCode=0 Jul 24 13:39:09.737681 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:09.737675 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb" event={"ID":"6c84771f-0639-4895-aa2e-8004a4b5df08","Type":"ContainerDied","Data":"a9d2a03b636c4fa4ce8e26bdfe6640cc2d1e8522ce0bb98ca6d9e9c87212c0b0"} Jul 24 13:39:12.752713 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:12.752676 2579 generic.go:356] "Generic (PLEG): container finished" podID="6c84771f-0639-4895-aa2e-8004a4b5df08" containerID="2d067e71c3d24f5402937cfc7de4b23f3618bd7f937358ba71273d869a314472" exitCode=0 Jul 24 13:39:12.753115 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:12.752730 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb" event={"ID":"6c84771f-0639-4895-aa2e-8004a4b5df08","Type":"ContainerDied","Data":"2d067e71c3d24f5402937cfc7de4b23f3618bd7f937358ba71273d869a314472"} Jul 24 13:39:19.205371 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:19.205346 2579 kubelet.go:1648] "Image garbage collection succeeded" Jul 24 13:39:19.784568 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:19.784528 2579 generic.go:356] "Generic (PLEG): container finished" podID="6c84771f-0639-4895-aa2e-8004a4b5df08" containerID="6a5f7a0d0047f4242fa0dd5f30893fb9d06a7cc82d9b3ecb861d60708eede0e9" exitCode=0 Jul 24 13:39:19.784765 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:19.784577 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb" event={"ID":"6c84771f-0639-4895-aa2e-8004a4b5df08","Type":"ContainerDied","Data":"6a5f7a0d0047f4242fa0dd5f30893fb9d06a7cc82d9b3ecb861d60708eede0e9"} Jul 24 13:39:20.909797 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:20.909773 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb" Jul 24 13:39:20.993228 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:20.993193 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92bth\" (UniqueName: \"kubernetes.io/projected/6c84771f-0639-4895-aa2e-8004a4b5df08-kube-api-access-92bth\") pod \"6c84771f-0639-4895-aa2e-8004a4b5df08\" (UID: \"6c84771f-0639-4895-aa2e-8004a4b5df08\") " Jul 24 13:39:20.993403 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:20.993255 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c84771f-0639-4895-aa2e-8004a4b5df08-bundle\") pod \"6c84771f-0639-4895-aa2e-8004a4b5df08\" (UID: \"6c84771f-0639-4895-aa2e-8004a4b5df08\") " Jul 24 13:39:20.993403 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:20.993283 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c84771f-0639-4895-aa2e-8004a4b5df08-util\") pod \"6c84771f-0639-4895-aa2e-8004a4b5df08\" (UID: \"6c84771f-0639-4895-aa2e-8004a4b5df08\") " Jul 24 13:39:20.993896 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:20.993868 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c84771f-0639-4895-aa2e-8004a4b5df08-bundle" (OuterVolumeSpecName: "bundle") pod "6c84771f-0639-4895-aa2e-8004a4b5df08" (UID: "6c84771f-0639-4895-aa2e-8004a4b5df08"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:39:20.995535 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:20.995506 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c84771f-0639-4895-aa2e-8004a4b5df08-kube-api-access-92bth" (OuterVolumeSpecName: "kube-api-access-92bth") pod "6c84771f-0639-4895-aa2e-8004a4b5df08" (UID: "6c84771f-0639-4895-aa2e-8004a4b5df08"). InnerVolumeSpecName "kube-api-access-92bth". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:39:20.997496 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:20.997474 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c84771f-0639-4895-aa2e-8004a4b5df08-util" (OuterVolumeSpecName: "util") pod "6c84771f-0639-4895-aa2e-8004a4b5df08" (UID: "6c84771f-0639-4895-aa2e-8004a4b5df08"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:39:21.094092 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:21.094050 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c84771f-0639-4895-aa2e-8004a4b5df08-bundle\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:39:21.094092 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:21.094077 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c84771f-0639-4895-aa2e-8004a4b5df08-util\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:39:21.094092 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:21.094086 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92bth\" (UniqueName: \"kubernetes.io/projected/6c84771f-0639-4895-aa2e-8004a4b5df08-kube-api-access-92bth\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:39:21.794420 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:21.794388 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb" Jul 24 13:39:21.794587 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:21.794392 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t78gb" event={"ID":"6c84771f-0639-4895-aa2e-8004a4b5df08","Type":"ContainerDied","Data":"8aeb3d9369918a75fa337ce325c157ec555e0320e76897d6ff01f63ef5022941"} Jul 24 13:39:21.794587 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:21.794509 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8aeb3d9369918a75fa337ce325c157ec555e0320e76897d6ff01f63ef5022941" Jul 24 13:39:24.054051 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:24.054018 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:25.394297 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:25.394267 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:25.938495 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:25.938464 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rh54s"] Jul 24 13:39:25.938750 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:25.938736 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c84771f-0639-4895-aa2e-8004a4b5df08" containerName="extract" Jul 24 13:39:25.938793 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:25.938754 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c84771f-0639-4895-aa2e-8004a4b5df08" containerName="extract" Jul 24 13:39:25.938793 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:25.938765 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c84771f-0639-4895-aa2e-8004a4b5df08" containerName="util" Jul 24 13:39:25.938793 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:25.938771 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c84771f-0639-4895-aa2e-8004a4b5df08" containerName="util" Jul 24 13:39:25.938793 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:25.938777 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c84771f-0639-4895-aa2e-8004a4b5df08" containerName="pull" Jul 24 13:39:25.938793 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:25.938783 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c84771f-0639-4895-aa2e-8004a4b5df08" containerName="pull" Jul 24 13:39:25.938934 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:25.938837 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c84771f-0639-4895-aa2e-8004a4b5df08" containerName="extract" Jul 24 13:39:25.941260 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:25.941242 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rh54s" Jul 24 13:39:25.944596 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:25.944570 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 24 13:39:25.944778 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:25.944760 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:39:25.944866 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:25.944830 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-pj2rk\"" Jul 24 13:39:25.957403 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:25.957381 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rh54s"] Jul 24 13:39:26.033396 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:26.033360 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e211cf65-98fd-4d94-a139-bd66c6034272-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rh54s\" (UID: \"e211cf65-98fd-4d94-a139-bd66c6034272\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rh54s" Jul 24 13:39:26.033567 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:26.033409 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk4lw\" (UniqueName: \"kubernetes.io/projected/e211cf65-98fd-4d94-a139-bd66c6034272-kube-api-access-qk4lw\") pod \"cert-manager-operator-controller-manager-55c444bf86-rh54s\" (UID: \"e211cf65-98fd-4d94-a139-bd66c6034272\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rh54s" Jul 24 13:39:26.134170 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:26.134135 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e211cf65-98fd-4d94-a139-bd66c6034272-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rh54s\" (UID: \"e211cf65-98fd-4d94-a139-bd66c6034272\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rh54s" Jul 24 13:39:26.134344 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:26.134180 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk4lw\" (UniqueName: \"kubernetes.io/projected/e211cf65-98fd-4d94-a139-bd66c6034272-kube-api-access-qk4lw\") pod \"cert-manager-operator-controller-manager-55c444bf86-rh54s\" (UID: \"e211cf65-98fd-4d94-a139-bd66c6034272\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rh54s" Jul 24 13:39:26.134563 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:26.134543 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e211cf65-98fd-4d94-a139-bd66c6034272-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rh54s\" (UID: \"e211cf65-98fd-4d94-a139-bd66c6034272\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rh54s" Jul 24 13:39:26.144713 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:26.144689 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk4lw\" (UniqueName: \"kubernetes.io/projected/e211cf65-98fd-4d94-a139-bd66c6034272-kube-api-access-qk4lw\") pod \"cert-manager-operator-controller-manager-55c444bf86-rh54s\" (UID: \"e211cf65-98fd-4d94-a139-bd66c6034272\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rh54s" Jul 24 13:39:26.250119 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:26.250029 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rh54s" Jul 24 13:39:26.388032 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:26.388007 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rh54s"] Jul 24 13:39:26.390927 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:39:26.390903 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode211cf65_98fd_4d94_a139_bd66c6034272.slice/crio-5fb9c9ba0eb4b2403c9d210064df7a1e97bbc33dd924e858a6ed669f70b7e3a6 WatchSource:0}: Error finding container 5fb9c9ba0eb4b2403c9d210064df7a1e97bbc33dd924e858a6ed669f70b7e3a6: Status 404 returned error can't find the container with id 5fb9c9ba0eb4b2403c9d210064df7a1e97bbc33dd924e858a6ed669f70b7e3a6 Jul 24 13:39:26.394330 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:26.394313 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:39:26.815374 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:26.815332 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rh54s" event={"ID":"e211cf65-98fd-4d94-a139-bd66c6034272","Type":"ContainerStarted","Data":"5fb9c9ba0eb4b2403c9d210064df7a1e97bbc33dd924e858a6ed669f70b7e3a6"} Jul 24 13:39:27.626141 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:27.626104 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:27.726522 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:27.726484 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:28.722872 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:28.722842 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:28.822647 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:28.822613 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:28.826429 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:28.826403 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rh54s" event={"ID":"e211cf65-98fd-4d94-a139-bd66c6034272","Type":"ContainerStarted","Data":"03c1757b3aed85606590bad91b8e606e4852233bab97b89de9bce88667674ef6"} Jul 24 13:39:28.855388 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:28.855340 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rh54s" podStartSLOduration=2.301651861 podStartE2EDuration="3.855324061s" totalImagesPullingTime="1.5536722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:39:28.852489824 +0000 UTC m=+310.975872223" watchObservedRunningTime="2026-07-24 13:39:28.855324061 +0000 UTC m=+310.978706458" Jul 24 13:39:28.917779 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:28.917743 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8vwkx"] Jul 24 13:39:28.921020 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:28.921002 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8vwkx" Jul 24 13:39:28.924129 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:28.924104 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 24 13:39:28.924256 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:28.924168 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-kpddj\"" Jul 24 13:39:28.924256 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:28.924197 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 24 13:39:28.930096 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:28.930068 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8vwkx"] Jul 24 13:39:29.058918 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:29.058889 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/295780a6-861e-4d1c-a291-d2b4ee6d497b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8vwkx\" (UID: \"295780a6-861e-4d1c-a291-d2b4ee6d497b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8vwkx" Jul 24 13:39:29.059100 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:29.058927 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkt56\" (UniqueName: \"kubernetes.io/projected/295780a6-861e-4d1c-a291-d2b4ee6d497b-kube-api-access-tkt56\") pod \"cert-manager-webhook-6486ffd997-8vwkx\" (UID: \"295780a6-861e-4d1c-a291-d2b4ee6d497b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8vwkx" Jul 24 13:39:29.159398 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:29.159358 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/295780a6-861e-4d1c-a291-d2b4ee6d497b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8vwkx\" (UID: \"295780a6-861e-4d1c-a291-d2b4ee6d497b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8vwkx" Jul 24 13:39:29.159586 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:29.159407 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkt56\" (UniqueName: \"kubernetes.io/projected/295780a6-861e-4d1c-a291-d2b4ee6d497b-kube-api-access-tkt56\") pod \"cert-manager-webhook-6486ffd997-8vwkx\" (UID: \"295780a6-861e-4d1c-a291-d2b4ee6d497b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8vwkx" Jul 24 13:39:29.183167 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:29.183137 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/295780a6-861e-4d1c-a291-d2b4ee6d497b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8vwkx\" (UID: \"295780a6-861e-4d1c-a291-d2b4ee6d497b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8vwkx" Jul 24 13:39:29.183398 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:29.183376 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkt56\" (UniqueName: \"kubernetes.io/projected/295780a6-861e-4d1c-a291-d2b4ee6d497b-kube-api-access-tkt56\") pod \"cert-manager-webhook-6486ffd997-8vwkx\" (UID: \"295780a6-861e-4d1c-a291-d2b4ee6d497b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8vwkx" Jul 24 13:39:29.224165 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:29.224129 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:29.237917 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:29.237881 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8vwkx" Jul 24 13:39:29.352473 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:29.352443 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:29.383819 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:29.383799 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8vwkx"] Jul 24 13:39:29.385247 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:39:29.385221 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod295780a6_861e_4d1c_a291_d2b4ee6d497b.slice/crio-148f50e2210bcbb76c84c7871e554837a0ce04b1055259ff27ec279a19670e06 WatchSource:0}: Error finding container 148f50e2210bcbb76c84c7871e554837a0ce04b1055259ff27ec279a19670e06: Status 404 returned error can't find the container with id 148f50e2210bcbb76c84c7871e554837a0ce04b1055259ff27ec279a19670e06 Jul 24 13:39:29.831499 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:29.831465 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8vwkx" event={"ID":"295780a6-861e-4d1c-a291-d2b4ee6d497b","Type":"ContainerStarted","Data":"148f50e2210bcbb76c84c7871e554837a0ce04b1055259ff27ec279a19670e06"} Jul 24 13:39:30.639523 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:30.639484 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs"] Jul 24 13:39:30.643317 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:30.643295 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs" Jul 24 13:39:30.646220 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:30.646135 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 13:39:30.647556 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:30.647523 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-t7rb5\"" Jul 24 13:39:30.647686 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:30.647560 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 13:39:30.653181 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:30.653043 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs"] Jul 24 13:39:30.773815 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:30.773723 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/411cbf13-4f17-4edb-baef-a993b55701b9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs\" (UID: \"411cbf13-4f17-4edb-baef-a993b55701b9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs" Jul 24 13:39:30.774093 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:30.773873 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/411cbf13-4f17-4edb-baef-a993b55701b9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs\" (UID: \"411cbf13-4f17-4edb-baef-a993b55701b9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs" Jul 24 13:39:30.774093 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:30.773931 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfpvg\" (UniqueName: \"kubernetes.io/projected/411cbf13-4f17-4edb-baef-a993b55701b9-kube-api-access-gfpvg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs\" (UID: \"411cbf13-4f17-4edb-baef-a993b55701b9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs" Jul 24 13:39:30.874784 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:30.874748 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/411cbf13-4f17-4edb-baef-a993b55701b9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs\" (UID: \"411cbf13-4f17-4edb-baef-a993b55701b9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs" Jul 24 13:39:30.875234 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:30.874814 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfpvg\" (UniqueName: \"kubernetes.io/projected/411cbf13-4f17-4edb-baef-a993b55701b9-kube-api-access-gfpvg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs\" (UID: \"411cbf13-4f17-4edb-baef-a993b55701b9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs" Jul 24 13:39:30.875234 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:30.874881 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/411cbf13-4f17-4edb-baef-a993b55701b9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs\" (UID: \"411cbf13-4f17-4edb-baef-a993b55701b9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs" Jul 24 13:39:30.875234 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:30.875206 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/411cbf13-4f17-4edb-baef-a993b55701b9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs\" (UID: \"411cbf13-4f17-4edb-baef-a993b55701b9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs" Jul 24 13:39:30.875234 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:30.875222 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/411cbf13-4f17-4edb-baef-a993b55701b9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs\" (UID: \"411cbf13-4f17-4edb-baef-a993b55701b9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs" Jul 24 13:39:30.883834 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:30.883810 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfpvg\" (UniqueName: \"kubernetes.io/projected/411cbf13-4f17-4edb-baef-a993b55701b9-kube-api-access-gfpvg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs\" (UID: \"411cbf13-4f17-4edb-baef-a993b55701b9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs" Jul 24 13:39:30.942063 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:30.941952 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:30.956021 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:30.955974 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs" Jul 24 13:39:31.106117 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:31.106076 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-pl7jm"] Jul 24 13:39:31.110394 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:31.110374 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pl7jm" Jul 24 13:39:31.113937 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:31.113289 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-pt295\"" Jul 24 13:39:31.117353 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:31.117004 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs"] Jul 24 13:39:31.118584 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:31.118565 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-pl7jm"] Jul 24 13:39:31.119218 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:39:31.119193 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod411cbf13_4f17_4edb_baef_a993b55701b9.slice/crio-f0cc3883ef85d8eadf65fbd63680ba97a8f4de21f17f2e5e234e3d6410a8ccc3 WatchSource:0}: Error finding container f0cc3883ef85d8eadf65fbd63680ba97a8f4de21f17f2e5e234e3d6410a8ccc3: Status 404 returned error can't find the container with id f0cc3883ef85d8eadf65fbd63680ba97a8f4de21f17f2e5e234e3d6410a8ccc3 Jul 24 13:39:31.132158 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:31.132131 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:31.176336 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:31.176311 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d6bf095-5d0e-4740-baca-9e63afd61a23-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-pl7jm\" (UID: \"9d6bf095-5d0e-4740-baca-9e63afd61a23\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pl7jm" Jul 24 13:39:31.176463 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:31.176447 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlhxf\" (UniqueName: \"kubernetes.io/projected/9d6bf095-5d0e-4740-baca-9e63afd61a23-kube-api-access-jlhxf\") pod \"cert-manager-cainjector-559cdd8c87-pl7jm\" (UID: \"9d6bf095-5d0e-4740-baca-9e63afd61a23\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pl7jm" Jul 24 13:39:31.276893 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:31.276865 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlhxf\" (UniqueName: \"kubernetes.io/projected/9d6bf095-5d0e-4740-baca-9e63afd61a23-kube-api-access-jlhxf\") pod \"cert-manager-cainjector-559cdd8c87-pl7jm\" (UID: \"9d6bf095-5d0e-4740-baca-9e63afd61a23\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pl7jm" Jul 24 13:39:31.277066 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:31.276914 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d6bf095-5d0e-4740-baca-9e63afd61a23-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-pl7jm\" (UID: \"9d6bf095-5d0e-4740-baca-9e63afd61a23\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pl7jm" Jul 24 13:39:31.285975 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:31.285947 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d6bf095-5d0e-4740-baca-9e63afd61a23-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-pl7jm\" (UID: \"9d6bf095-5d0e-4740-baca-9e63afd61a23\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pl7jm" Jul 24 13:39:31.286145 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:31.286066 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlhxf\" (UniqueName: \"kubernetes.io/projected/9d6bf095-5d0e-4740-baca-9e63afd61a23-kube-api-access-jlhxf\") pod \"cert-manager-cainjector-559cdd8c87-pl7jm\" (UID: \"9d6bf095-5d0e-4740-baca-9e63afd61a23\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pl7jm" Jul 24 13:39:31.431919 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:31.431878 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pl7jm" Jul 24 13:39:31.842939 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:31.842905 2579 generic.go:356] "Generic (PLEG): container finished" podID="411cbf13-4f17-4edb-baef-a993b55701b9" containerID="1234a39845bf5214c2eb1bc511777f5ebd0f3d83a7c65bf9b249e9f5891fea63" exitCode=0 Jul 24 13:39:31.843141 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:31.842962 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs" event={"ID":"411cbf13-4f17-4edb-baef-a993b55701b9","Type":"ContainerDied","Data":"1234a39845bf5214c2eb1bc511777f5ebd0f3d83a7c65bf9b249e9f5891fea63"} Jul 24 13:39:31.843141 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:31.843029 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs" event={"ID":"411cbf13-4f17-4edb-baef-a993b55701b9","Type":"ContainerStarted","Data":"f0cc3883ef85d8eadf65fbd63680ba97a8f4de21f17f2e5e234e3d6410a8ccc3"} Jul 24 13:39:32.269207 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:32.269186 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-pl7jm"] Jul 24 13:39:32.270831 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:39:32.270805 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d6bf095_5d0e_4740_baca_9e63afd61a23.slice/crio-dbf834bb4a44de5ea9eaa0111f508b90f560e998ffec67d52735c94931cd0344 WatchSource:0}: Error finding container dbf834bb4a44de5ea9eaa0111f508b90f560e998ffec67d52735c94931cd0344: Status 404 returned error can't find the container with id dbf834bb4a44de5ea9eaa0111f508b90f560e998ffec67d52735c94931cd0344 Jul 24 13:39:32.849278 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:32.849219 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pl7jm" event={"ID":"9d6bf095-5d0e-4740-baca-9e63afd61a23","Type":"ContainerStarted","Data":"fcaf0ca8e3a2cc52fbba7a36f0802b510ebcbb53b4f2688815d6e487fa301031"} Jul 24 13:39:32.849278 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:32.849271 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pl7jm" event={"ID":"9d6bf095-5d0e-4740-baca-9e63afd61a23","Type":"ContainerStarted","Data":"dbf834bb4a44de5ea9eaa0111f508b90f560e998ffec67d52735c94931cd0344"} Jul 24 13:39:32.851051 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:32.851022 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8vwkx" event={"ID":"295780a6-861e-4d1c-a291-d2b4ee6d497b","Type":"ContainerStarted","Data":"54750644b490170cac20a59b7c9d8c75f60492d7bd99c4b96dfc9ecf6b4c7a5a"} Jul 24 13:39:32.851179 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:32.851074 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8vwkx" Jul 24 13:39:32.866967 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:32.866919 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pl7jm" podStartSLOduration=1.866901634 podStartE2EDuration="1.866901634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:39:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:39:32.864586856 +0000 UTC m=+314.987969246" watchObservedRunningTime="2026-07-24 13:39:32.866901634 +0000 UTC m=+314.990284034" Jul 24 13:39:32.882691 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:32.882620 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-8vwkx" podStartSLOduration=2.066639018 podStartE2EDuration="4.882597013s" totalImagesPullingTime="2.815957995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:39:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:39:32.88028536 +0000 UTC m=+315.003667821" watchObservedRunningTime="2026-07-24 13:39:32.882597013 +0000 UTC m=+315.005979414" Jul 24 13:39:33.857080 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:33.857044 2579 generic.go:356] "Generic (PLEG): container finished" podID="411cbf13-4f17-4edb-baef-a993b55701b9" containerID="33067175fb31522d24d26cb1fad37ec6776ee915297df8ca085c14bb27911727" exitCode=0 Jul 24 13:39:33.857468 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:33.857142 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs" event={"ID":"411cbf13-4f17-4edb-baef-a993b55701b9","Type":"ContainerDied","Data":"33067175fb31522d24d26cb1fad37ec6776ee915297df8ca085c14bb27911727"} Jul 24 13:39:34.412272 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:34.412240 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:34.472762 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:34.472728 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:34.863892 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:34.863852 2579 generic.go:356] "Generic (PLEG): container finished" podID="411cbf13-4f17-4edb-baef-a993b55701b9" containerID="23f8155c6979612a3df726d9d34c00996da001b1c594b26030cc7f95ff75ff18" exitCode=0 Jul 24 13:39:34.864262 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:34.863903 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs" event={"ID":"411cbf13-4f17-4edb-baef-a993b55701b9","Type":"ContainerDied","Data":"23f8155c6979612a3df726d9d34c00996da001b1c594b26030cc7f95ff75ff18"} Jul 24 13:39:35.999423 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:35.999401 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs" Jul 24 13:39:36.122703 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:36.122616 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/411cbf13-4f17-4edb-baef-a993b55701b9-bundle\") pod \"411cbf13-4f17-4edb-baef-a993b55701b9\" (UID: \"411cbf13-4f17-4edb-baef-a993b55701b9\") " Jul 24 13:39:36.122703 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:36.122673 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfpvg\" (UniqueName: \"kubernetes.io/projected/411cbf13-4f17-4edb-baef-a993b55701b9-kube-api-access-gfpvg\") pod \"411cbf13-4f17-4edb-baef-a993b55701b9\" (UID: \"411cbf13-4f17-4edb-baef-a993b55701b9\") " Jul 24 13:39:36.122876 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:36.122719 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/411cbf13-4f17-4edb-baef-a993b55701b9-util\") pod \"411cbf13-4f17-4edb-baef-a993b55701b9\" (UID: \"411cbf13-4f17-4edb-baef-a993b55701b9\") " Jul 24 13:39:36.123179 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:36.123128 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411cbf13-4f17-4edb-baef-a993b55701b9-bundle" (OuterVolumeSpecName: "bundle") pod "411cbf13-4f17-4edb-baef-a993b55701b9" (UID: "411cbf13-4f17-4edb-baef-a993b55701b9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:39:36.125063 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:36.125039 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/411cbf13-4f17-4edb-baef-a993b55701b9-kube-api-access-gfpvg" (OuterVolumeSpecName: "kube-api-access-gfpvg") pod "411cbf13-4f17-4edb-baef-a993b55701b9" (UID: "411cbf13-4f17-4edb-baef-a993b55701b9"). InnerVolumeSpecName "kube-api-access-gfpvg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:39:36.128003 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:36.127957 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411cbf13-4f17-4edb-baef-a993b55701b9-util" (OuterVolumeSpecName: "util") pod "411cbf13-4f17-4edb-baef-a993b55701b9" (UID: "411cbf13-4f17-4edb-baef-a993b55701b9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:39:36.223347 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:36.223308 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/411cbf13-4f17-4edb-baef-a993b55701b9-util\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:39:36.223347 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:36.223341 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/411cbf13-4f17-4edb-baef-a993b55701b9-bundle\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:39:36.223347 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:36.223350 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfpvg\" (UniqueName: \"kubernetes.io/projected/411cbf13-4f17-4edb-baef-a993b55701b9-kube-api-access-gfpvg\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:39:36.874761 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:36.874730 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs" event={"ID":"411cbf13-4f17-4edb-baef-a993b55701b9","Type":"ContainerDied","Data":"f0cc3883ef85d8eadf65fbd63680ba97a8f4de21f17f2e5e234e3d6410a8ccc3"} Jul 24 13:39:36.874761 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:36.874756 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsfzrs" Jul 24 13:39:36.874969 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:36.874781 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0cc3883ef85d8eadf65fbd63680ba97a8f4de21f17f2e5e234e3d6410a8ccc3" Jul 24 13:39:38.859821 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:38.859794 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8vwkx" Jul 24 13:39:39.332128 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:39.332091 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:40.438768 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:40.438720 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:40.537828 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:40.537785 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:47.470002 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:47.469946 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:47.549690 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:47.549660 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:50.319337 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.319303 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x"] Jul 24 13:39:50.319729 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.319588 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="411cbf13-4f17-4edb-baef-a993b55701b9" containerName="util" Jul 24 13:39:50.319729 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.319599 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="411cbf13-4f17-4edb-baef-a993b55701b9" containerName="util" Jul 24 13:39:50.319729 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.319608 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="411cbf13-4f17-4edb-baef-a993b55701b9" containerName="extract" Jul 24 13:39:50.319729 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.319614 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="411cbf13-4f17-4edb-baef-a993b55701b9" containerName="extract" Jul 24 13:39:50.319729 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.319627 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="411cbf13-4f17-4edb-baef-a993b55701b9" containerName="pull" Jul 24 13:39:50.319729 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.319632 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="411cbf13-4f17-4edb-baef-a993b55701b9" containerName="pull" Jul 24 13:39:50.319729 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.319699 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="411cbf13-4f17-4edb-baef-a993b55701b9" containerName="extract" Jul 24 13:39:50.379087 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.379053 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x"] Jul 24 13:39:50.379234 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.379170 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x" Jul 24 13:39:50.382589 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.382560 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-t7rb5\"" Jul 24 13:39:50.382737 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.382604 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 13:39:50.383712 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.383691 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 13:39:50.453743 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.453702 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfpvv\" (UniqueName: \"kubernetes.io/projected/263184b2-30fe-4b1a-9090-dfb3771effd0-kube-api-access-hfpvv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x\" (UID: \"263184b2-30fe-4b1a-9090-dfb3771effd0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x" Jul 24 13:39:50.453914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.453835 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/263184b2-30fe-4b1a-9090-dfb3771effd0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x\" (UID: \"263184b2-30fe-4b1a-9090-dfb3771effd0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x" Jul 24 13:39:50.453914 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.453883 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/263184b2-30fe-4b1a-9090-dfb3771effd0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x\" (UID: \"263184b2-30fe-4b1a-9090-dfb3771effd0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x" Jul 24 13:39:50.555164 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.555128 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/263184b2-30fe-4b1a-9090-dfb3771effd0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x\" (UID: \"263184b2-30fe-4b1a-9090-dfb3771effd0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x" Jul 24 13:39:50.555316 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.555173 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfpvv\" (UniqueName: \"kubernetes.io/projected/263184b2-30fe-4b1a-9090-dfb3771effd0-kube-api-access-hfpvv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x\" (UID: \"263184b2-30fe-4b1a-9090-dfb3771effd0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x" Jul 24 13:39:50.555316 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.555226 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/263184b2-30fe-4b1a-9090-dfb3771effd0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x\" (UID: \"263184b2-30fe-4b1a-9090-dfb3771effd0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x" Jul 24 13:39:50.555563 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.555547 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/263184b2-30fe-4b1a-9090-dfb3771effd0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x\" (UID: \"263184b2-30fe-4b1a-9090-dfb3771effd0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x" Jul 24 13:39:50.555603 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.555548 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/263184b2-30fe-4b1a-9090-dfb3771effd0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x\" (UID: \"263184b2-30fe-4b1a-9090-dfb3771effd0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x" Jul 24 13:39:50.564142 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.564119 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfpvv\" (UniqueName: \"kubernetes.io/projected/263184b2-30fe-4b1a-9090-dfb3771effd0-kube-api-access-hfpvv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x\" (UID: \"263184b2-30fe-4b1a-9090-dfb3771effd0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x" Jul 24 13:39:50.644445 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.644347 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:50.691506 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.691478 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x" Jul 24 13:39:50.841259 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.841149 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x"] Jul 24 13:39:50.843398 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:39:50.843362 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod263184b2_30fe_4b1a_9090_dfb3771effd0.slice/crio-1c3b36a983a7658ae0b349585be5a591f6ca64e686db9505e9e03e47a315c9ba WatchSource:0}: Error finding container 1c3b36a983a7658ae0b349585be5a591f6ca64e686db9505e9e03e47a315c9ba: Status 404 returned error can't find the container with id 1c3b36a983a7658ae0b349585be5a591f6ca64e686db9505e9e03e47a315c9ba Jul 24 13:39:50.933730 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:50.933697 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x" event={"ID":"263184b2-30fe-4b1a-9090-dfb3771effd0","Type":"ContainerStarted","Data":"1c3b36a983a7658ae0b349585be5a591f6ca64e686db9505e9e03e47a315c9ba"} Jul 24 13:39:51.939566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:51.939466 2579 generic.go:356] "Generic (PLEG): container finished" podID="263184b2-30fe-4b1a-9090-dfb3771effd0" containerID="895d2f2ae8e0653e3ff49f064f3bb432fdf06d13c5dbe2c376cb125749eedd80" exitCode=0 Jul 24 13:39:51.939566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:51.939547 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x" event={"ID":"263184b2-30fe-4b1a-9090-dfb3771effd0","Type":"ContainerDied","Data":"895d2f2ae8e0653e3ff49f064f3bb432fdf06d13c5dbe2c376cb125749eedd80"} Jul 24 13:39:52.946072 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:52.946024 2579 generic.go:356] "Generic (PLEG): container finished" podID="263184b2-30fe-4b1a-9090-dfb3771effd0" containerID="2ae84854b2bc581993d08c10f3f000463e1ce64c40e4cf8a3268a96273bb4da9" exitCode=0 Jul 24 13:39:52.946465 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:52.946096 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x" event={"ID":"263184b2-30fe-4b1a-9090-dfb3771effd0","Type":"ContainerDied","Data":"2ae84854b2bc581993d08c10f3f000463e1ce64c40e4cf8a3268a96273bb4da9"} Jul 24 13:39:53.952769 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:53.952734 2579 generic.go:356] "Generic (PLEG): container finished" podID="263184b2-30fe-4b1a-9090-dfb3771effd0" containerID="04679bd0f0a523b5158bf6fcd2767d3508f36108fd4d6dde30ee4969ae8789f1" exitCode=0 Jul 24 13:39:53.953151 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:53.952821 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x" event={"ID":"263184b2-30fe-4b1a-9090-dfb3771effd0","Type":"ContainerDied","Data":"04679bd0f0a523b5158bf6fcd2767d3508f36108fd4d6dde30ee4969ae8789f1"} Jul 24 13:39:55.085672 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:55.085650 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x" Jul 24 13:39:55.185822 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:55.185793 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/263184b2-30fe-4b1a-9090-dfb3771effd0-util\") pod \"263184b2-30fe-4b1a-9090-dfb3771effd0\" (UID: \"263184b2-30fe-4b1a-9090-dfb3771effd0\") " Jul 24 13:39:55.186015 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:55.185843 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfpvv\" (UniqueName: \"kubernetes.io/projected/263184b2-30fe-4b1a-9090-dfb3771effd0-kube-api-access-hfpvv\") pod \"263184b2-30fe-4b1a-9090-dfb3771effd0\" (UID: \"263184b2-30fe-4b1a-9090-dfb3771effd0\") " Jul 24 13:39:55.186015 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:55.185878 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/263184b2-30fe-4b1a-9090-dfb3771effd0-bundle\") pod \"263184b2-30fe-4b1a-9090-dfb3771effd0\" (UID: \"263184b2-30fe-4b1a-9090-dfb3771effd0\") " Jul 24 13:39:55.186632 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:55.186608 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/263184b2-30fe-4b1a-9090-dfb3771effd0-bundle" (OuterVolumeSpecName: "bundle") pod "263184b2-30fe-4b1a-9090-dfb3771effd0" (UID: "263184b2-30fe-4b1a-9090-dfb3771effd0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:39:55.188196 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:55.188175 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/263184b2-30fe-4b1a-9090-dfb3771effd0-kube-api-access-hfpvv" (OuterVolumeSpecName: "kube-api-access-hfpvv") pod "263184b2-30fe-4b1a-9090-dfb3771effd0" (UID: "263184b2-30fe-4b1a-9090-dfb3771effd0"). InnerVolumeSpecName "kube-api-access-hfpvv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:39:55.191150 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:55.191129 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/263184b2-30fe-4b1a-9090-dfb3771effd0-util" (OuterVolumeSpecName: "util") pod "263184b2-30fe-4b1a-9090-dfb3771effd0" (UID: "263184b2-30fe-4b1a-9090-dfb3771effd0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:39:55.286650 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:55.286572 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/263184b2-30fe-4b1a-9090-dfb3771effd0-bundle\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:39:55.286650 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:55.286596 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/263184b2-30fe-4b1a-9090-dfb3771effd0-util\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:39:55.286650 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:55.286604 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfpvv\" (UniqueName: \"kubernetes.io/projected/263184b2-30fe-4b1a-9090-dfb3771effd0-kube-api-access-hfpvv\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:39:55.964216 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:55.964174 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x" event={"ID":"263184b2-30fe-4b1a-9090-dfb3771effd0","Type":"ContainerDied","Data":"1c3b36a983a7658ae0b349585be5a591f6ca64e686db9505e9e03e47a315c9ba"} Jul 24 13:39:55.964384 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:55.964261 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c3b36a983a7658ae0b349585be5a591f6ca64e686db9505e9e03e47a315c9ba" Jul 24 13:39:55.964384 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:39:55.964193 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzp9x" Jul 24 13:40:02.250554 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:02.250517 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:02.356898 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:02.356842 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:02.699388 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:02.699338 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:03.129557 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.129526 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:03.226723 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.226685 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:03.310905 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.310875 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf"] Jul 24 13:40:03.311313 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.311169 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="263184b2-30fe-4b1a-9090-dfb3771effd0" containerName="pull" Jul 24 13:40:03.311313 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.311179 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="263184b2-30fe-4b1a-9090-dfb3771effd0" containerName="pull" Jul 24 13:40:03.311313 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.311200 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="263184b2-30fe-4b1a-9090-dfb3771effd0" containerName="extract" Jul 24 13:40:03.311313 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.311205 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="263184b2-30fe-4b1a-9090-dfb3771effd0" containerName="extract" Jul 24 13:40:03.311313 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.311216 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="263184b2-30fe-4b1a-9090-dfb3771effd0" containerName="util" Jul 24 13:40:03.311313 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.311222 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="263184b2-30fe-4b1a-9090-dfb3771effd0" containerName="util" Jul 24 13:40:03.311313 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.311266 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="263184b2-30fe-4b1a-9090-dfb3771effd0" containerName="extract" Jul 24 13:40:03.341492 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.341463 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf"] Jul 24 13:40:03.341655 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.341596 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:03.344478 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.344455 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 24 13:40:03.344625 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.344554 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 24 13:40:03.344873 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.344853 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fjml2\"" Jul 24 13:40:03.344972 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.344949 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 24 13:40:03.345195 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.345178 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 24 13:40:03.453048 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.452935 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba47ec72-ddad-41a3-906d-61eae421e7ab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85fcbb566-fmjsf\" (UID: \"ba47ec72-ddad-41a3-906d-61eae421e7ab\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:03.453297 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.453274 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba47ec72-ddad-41a3-906d-61eae421e7ab-tmp\") pod \"opendatahub-operator-controller-manager-85fcbb566-fmjsf\" (UID: \"ba47ec72-ddad-41a3-906d-61eae421e7ab\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:03.453440 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.453419 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba47ec72-ddad-41a3-906d-61eae421e7ab-webhook-cert\") pod \"opendatahub-operator-controller-manager-85fcbb566-fmjsf\" (UID: \"ba47ec72-ddad-41a3-906d-61eae421e7ab\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:03.453569 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.453459 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82b9q\" (UniqueName: \"kubernetes.io/projected/ba47ec72-ddad-41a3-906d-61eae421e7ab-kube-api-access-82b9q\") pod \"opendatahub-operator-controller-manager-85fcbb566-fmjsf\" (UID: \"ba47ec72-ddad-41a3-906d-61eae421e7ab\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:03.453569 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.453535 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba47ec72-ddad-41a3-906d-61eae421e7ab-manifests\") pod \"opendatahub-operator-controller-manager-85fcbb566-fmjsf\" (UID: \"ba47ec72-ddad-41a3-906d-61eae421e7ab\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:03.512521 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.512485 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:03.554284 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.554252 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82b9q\" (UniqueName: \"kubernetes.io/projected/ba47ec72-ddad-41a3-906d-61eae421e7ab-kube-api-access-82b9q\") pod \"opendatahub-operator-controller-manager-85fcbb566-fmjsf\" (UID: \"ba47ec72-ddad-41a3-906d-61eae421e7ab\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:03.554445 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.554351 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba47ec72-ddad-41a3-906d-61eae421e7ab-manifests\") pod \"opendatahub-operator-controller-manager-85fcbb566-fmjsf\" (UID: \"ba47ec72-ddad-41a3-906d-61eae421e7ab\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:03.554445 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.554418 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba47ec72-ddad-41a3-906d-61eae421e7ab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85fcbb566-fmjsf\" (UID: \"ba47ec72-ddad-41a3-906d-61eae421e7ab\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:03.554573 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.554446 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba47ec72-ddad-41a3-906d-61eae421e7ab-tmp\") pod \"opendatahub-operator-controller-manager-85fcbb566-fmjsf\" (UID: \"ba47ec72-ddad-41a3-906d-61eae421e7ab\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:03.554573 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.554487 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba47ec72-ddad-41a3-906d-61eae421e7ab-webhook-cert\") pod \"opendatahub-operator-controller-manager-85fcbb566-fmjsf\" (UID: \"ba47ec72-ddad-41a3-906d-61eae421e7ab\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:03.554849 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.554823 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba47ec72-ddad-41a3-906d-61eae421e7ab-manifests\") pod \"opendatahub-operator-controller-manager-85fcbb566-fmjsf\" (UID: \"ba47ec72-ddad-41a3-906d-61eae421e7ab\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:03.557127 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.557105 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba47ec72-ddad-41a3-906d-61eae421e7ab-tmp\") pod \"opendatahub-operator-controller-manager-85fcbb566-fmjsf\" (UID: \"ba47ec72-ddad-41a3-906d-61eae421e7ab\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:03.557451 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.557426 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba47ec72-ddad-41a3-906d-61eae421e7ab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85fcbb566-fmjsf\" (UID: \"ba47ec72-ddad-41a3-906d-61eae421e7ab\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:03.557514 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.557426 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba47ec72-ddad-41a3-906d-61eae421e7ab-webhook-cert\") pod \"opendatahub-operator-controller-manager-85fcbb566-fmjsf\" (UID: \"ba47ec72-ddad-41a3-906d-61eae421e7ab\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:03.566220 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.566200 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82b9q\" (UniqueName: \"kubernetes.io/projected/ba47ec72-ddad-41a3-906d-61eae421e7ab-kube-api-access-82b9q\") pod \"opendatahub-operator-controller-manager-85fcbb566-fmjsf\" (UID: \"ba47ec72-ddad-41a3-906d-61eae421e7ab\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:03.636665 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.636632 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:03.651662 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.651628 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:03.764202 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.764157 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:03.823812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.823790 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf"] Jul 24 13:40:03.826402 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:40:03.826374 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba47ec72_ddad_41a3_906d_61eae421e7ab.slice/crio-9655d1fc3d1e084deca52a5f86a15c80e2f87548f08738df4686d21943026ecf WatchSource:0}: Error finding container 9655d1fc3d1e084deca52a5f86a15c80e2f87548f08738df4686d21943026ecf: Status 404 returned error can't find the container with id 9655d1fc3d1e084deca52a5f86a15c80e2f87548f08738df4686d21943026ecf Jul 24 13:40:03.878094 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:03.878062 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:04.000797 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:04.000711 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" event={"ID":"ba47ec72-ddad-41a3-906d-61eae421e7ab","Type":"ContainerStarted","Data":"9655d1fc3d1e084deca52a5f86a15c80e2f87548f08738df4686d21943026ecf"} Jul 24 13:40:04.066833 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:04.066796 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:07.018482 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:07.018399 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" event={"ID":"ba47ec72-ddad-41a3-906d-61eae421e7ab","Type":"ContainerStarted","Data":"86b0ec191903555d79178be0e9fbfae2fb271b5f59256db36e7ec4eb07a372e9"} Jul 24 13:40:08.025738 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:08.025703 2579 generic.go:356] "Generic (PLEG): container finished" podID="ba47ec72-ddad-41a3-906d-61eae421e7ab" containerID="86b0ec191903555d79178be0e9fbfae2fb271b5f59256db36e7ec4eb07a372e9" exitCode=0 Jul 24 13:40:08.026137 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:08.025791 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" event={"ID":"ba47ec72-ddad-41a3-906d-61eae421e7ab","Type":"ContainerDied","Data":"86b0ec191903555d79178be0e9fbfae2fb271b5f59256db36e7ec4eb07a372e9"} Jul 24 13:40:09.033065 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:09.032958 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" event={"ID":"ba47ec72-ddad-41a3-906d-61eae421e7ab","Type":"ContainerStarted","Data":"f99a60e9246f4bc5149467d3e1bc992b12020ea871154efeb5d4401fde9499f4"} Jul 24 13:40:09.033562 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:09.033222 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:09.078777 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:09.078724 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" podStartSLOduration=1.789333332 podStartE2EDuration="6.078707889s" totalImagesPullingTime="3.258794924s" totalInitContainerRuntime="1.030579633s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:40:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:40:09.076173273 +0000 UTC m=+351.199555671" watchObservedRunningTime="2026-07-24 13:40:09.078707889 +0000 UTC m=+351.202090416" Jul 24 13:40:09.106236 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:09.106191 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:09.208947 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:09.208908 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:14.336226 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:14.336184 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:14.429544 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:14.429511 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:20.039008 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:20.038968 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-85fcbb566-fmjsf" Jul 24 13:40:22.373135 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:22.373095 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:22.603304 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:22.603272 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:23.065568 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.065536 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:23.134638 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.134589 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-85dc78bf9-5v5rs"] Jul 24 13:40:23.140962 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.140929 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85dc78bf9-5v5rs" Jul 24 13:40:23.144744 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.144722 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-ljhn7\"" Jul 24 13:40:23.144943 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.144923 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 24 13:40:23.146125 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.146107 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 24 13:40:23.146238 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.146147 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 24 13:40:23.146238 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.146206 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 24 13:40:23.162955 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.162928 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85dc78bf9-5v5rs"] Jul 24 13:40:23.225412 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.225375 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p66tz\" (UniqueName: \"kubernetes.io/projected/ef5cbd04-8ca5-4930-82a3-a722b0ab36d3-kube-api-access-p66tz\") pod \"kube-auth-proxy-85dc78bf9-5v5rs\" (UID: \"ef5cbd04-8ca5-4930-82a3-a722b0ab36d3\") " pod="openshift-ingress/kube-auth-proxy-85dc78bf9-5v5rs" Jul 24 13:40:23.225601 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.225418 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef5cbd04-8ca5-4930-82a3-a722b0ab36d3-tmp\") pod \"kube-auth-proxy-85dc78bf9-5v5rs\" (UID: \"ef5cbd04-8ca5-4930-82a3-a722b0ab36d3\") " pod="openshift-ingress/kube-auth-proxy-85dc78bf9-5v5rs" Jul 24 13:40:23.225601 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.225534 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef5cbd04-8ca5-4930-82a3-a722b0ab36d3-tls-certs\") pod \"kube-auth-proxy-85dc78bf9-5v5rs\" (UID: \"ef5cbd04-8ca5-4930-82a3-a722b0ab36d3\") " pod="openshift-ingress/kube-auth-proxy-85dc78bf9-5v5rs" Jul 24 13:40:23.270760 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.270715 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:23.289772 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.289738 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-b4ln4"] Jul 24 13:40:23.292497 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.292469 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-b4ln4" Jul 24 13:40:23.296052 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.296027 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-57q9d\"" Jul 24 13:40:23.296286 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.296271 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 24 13:40:23.307545 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.307522 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-b4ln4"] Jul 24 13:40:23.327647 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.327557 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p66tz\" (UniqueName: \"kubernetes.io/projected/ef5cbd04-8ca5-4930-82a3-a722b0ab36d3-kube-api-access-p66tz\") pod \"kube-auth-proxy-85dc78bf9-5v5rs\" (UID: \"ef5cbd04-8ca5-4930-82a3-a722b0ab36d3\") " pod="openshift-ingress/kube-auth-proxy-85dc78bf9-5v5rs" Jul 24 13:40:23.327900 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.327875 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef5cbd04-8ca5-4930-82a3-a722b0ab36d3-tmp\") pod \"kube-auth-proxy-85dc78bf9-5v5rs\" (UID: \"ef5cbd04-8ca5-4930-82a3-a722b0ab36d3\") " pod="openshift-ingress/kube-auth-proxy-85dc78bf9-5v5rs" Jul 24 13:40:23.328118 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.328101 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef5cbd04-8ca5-4930-82a3-a722b0ab36d3-tls-certs\") pod \"kube-auth-proxy-85dc78bf9-5v5rs\" (UID: \"ef5cbd04-8ca5-4930-82a3-a722b0ab36d3\") " pod="openshift-ingress/kube-auth-proxy-85dc78bf9-5v5rs" Jul 24 13:40:23.331413 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.331387 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef5cbd04-8ca5-4930-82a3-a722b0ab36d3-tmp\") pod \"kube-auth-proxy-85dc78bf9-5v5rs\" (UID: \"ef5cbd04-8ca5-4930-82a3-a722b0ab36d3\") " pod="openshift-ingress/kube-auth-proxy-85dc78bf9-5v5rs" Jul 24 13:40:23.331594 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.331577 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef5cbd04-8ca5-4930-82a3-a722b0ab36d3-tls-certs\") pod \"kube-auth-proxy-85dc78bf9-5v5rs\" (UID: \"ef5cbd04-8ca5-4930-82a3-a722b0ab36d3\") " pod="openshift-ingress/kube-auth-proxy-85dc78bf9-5v5rs" Jul 24 13:40:23.342372 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.342331 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p66tz\" (UniqueName: \"kubernetes.io/projected/ef5cbd04-8ca5-4930-82a3-a722b0ab36d3-kube-api-access-p66tz\") pod \"kube-auth-proxy-85dc78bf9-5v5rs\" (UID: \"ef5cbd04-8ca5-4930-82a3-a722b0ab36d3\") " pod="openshift-ingress/kube-auth-proxy-85dc78bf9-5v5rs" Jul 24 13:40:23.391287 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.391249 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:23.429404 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.429362 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg9xh\" (UniqueName: \"kubernetes.io/projected/24b46733-ac33-433b-bf13-17bbbbdfc040-kube-api-access-mg9xh\") pod \"odh-model-controller-7998c67dc4-b4ln4\" (UID: \"24b46733-ac33-433b-bf13-17bbbbdfc040\") " pod="opendatahub/odh-model-controller-7998c67dc4-b4ln4" Jul 24 13:40:23.429404 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.429411 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24b46733-ac33-433b-bf13-17bbbbdfc040-cert\") pod \"odh-model-controller-7998c67dc4-b4ln4\" (UID: \"24b46733-ac33-433b-bf13-17bbbbdfc040\") " pod="opendatahub/odh-model-controller-7998c67dc4-b4ln4" Jul 24 13:40:23.451267 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.451234 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85dc78bf9-5v5rs" Jul 24 13:40:23.530283 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.530254 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg9xh\" (UniqueName: \"kubernetes.io/projected/24b46733-ac33-433b-bf13-17bbbbdfc040-kube-api-access-mg9xh\") pod \"odh-model-controller-7998c67dc4-b4ln4\" (UID: \"24b46733-ac33-433b-bf13-17bbbbdfc040\") " pod="opendatahub/odh-model-controller-7998c67dc4-b4ln4" Jul 24 13:40:23.530459 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.530305 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24b46733-ac33-433b-bf13-17bbbbdfc040-cert\") pod \"odh-model-controller-7998c67dc4-b4ln4\" (UID: \"24b46733-ac33-433b-bf13-17bbbbdfc040\") " pod="opendatahub/odh-model-controller-7998c67dc4-b4ln4" Jul 24 13:40:23.530459 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:40:23.530443 2579 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 24 13:40:23.530580 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:40:23.530509 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24b46733-ac33-433b-bf13-17bbbbdfc040-cert podName:24b46733-ac33-433b-bf13-17bbbbdfc040 nodeName:}" failed. No retries permitted until 2026-07-24 13:40:24.03048516 +0000 UTC m=+366.153867553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/24b46733-ac33-433b-bf13-17bbbbdfc040-cert") pod "odh-model-controller-7998c67dc4-b4ln4" (UID: "24b46733-ac33-433b-bf13-17bbbbdfc040") : secret "odh-model-controller-webhook-cert" not found Jul 24 13:40:23.532105 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.532059 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:23.547420 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.547345 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg9xh\" (UniqueName: \"kubernetes.io/projected/24b46733-ac33-433b-bf13-17bbbbdfc040-kube-api-access-mg9xh\") pod \"odh-model-controller-7998c67dc4-b4ln4\" (UID: \"24b46733-ac33-433b-bf13-17bbbbdfc040\") " pod="opendatahub/odh-model-controller-7998c67dc4-b4ln4" Jul 24 13:40:23.604828 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.604798 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85dc78bf9-5v5rs"] Jul 24 13:40:23.608116 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:40:23.608091 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef5cbd04_8ca5_4930_82a3_a722b0ab36d3.slice/crio-c850bb86b8219f03a9d3b1b706c192bd739139ca52e81c52c6a978953f47275a WatchSource:0}: Error finding container c850bb86b8219f03a9d3b1b706c192bd739139ca52e81c52c6a978953f47275a: Status 404 returned error can't find the container with id c850bb86b8219f03a9d3b1b706c192bd739139ca52e81c52c6a978953f47275a Jul 24 13:40:23.638942 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.638912 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:23.770873 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:23.770838 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:24.034588 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:24.034510 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24b46733-ac33-433b-bf13-17bbbbdfc040-cert\") pod \"odh-model-controller-7998c67dc4-b4ln4\" (UID: \"24b46733-ac33-433b-bf13-17bbbbdfc040\") " pod="opendatahub/odh-model-controller-7998c67dc4-b4ln4" Jul 24 13:40:24.037045 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:24.037018 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24b46733-ac33-433b-bf13-17bbbbdfc040-cert\") pod \"odh-model-controller-7998c67dc4-b4ln4\" (UID: \"24b46733-ac33-433b-bf13-17bbbbdfc040\") " pod="opendatahub/odh-model-controller-7998c67dc4-b4ln4" Jul 24 13:40:24.108095 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:24.108057 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85dc78bf9-5v5rs" event={"ID":"ef5cbd04-8ca5-4930-82a3-a722b0ab36d3","Type":"ContainerStarted","Data":"c850bb86b8219f03a9d3b1b706c192bd739139ca52e81c52c6a978953f47275a"} Jul 24 13:40:24.202853 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:24.202806 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-b4ln4" Jul 24 13:40:24.401369 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:24.401335 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:24.487309 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:24.487243 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-b4ln4"] Jul 24 13:40:24.489253 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:24.489228 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hfvdp"] Jul 24 13:40:24.493783 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:24.493762 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hfvdp" Jul 24 13:40:24.526087 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:40:24.526048 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"model-serving-api-tls\" is forbidden: User \"system:node:ip-10-0-137-60.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-60.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" type="*v1.Secret" Jul 24 13:40:24.526282 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:40:24.526252 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"model-serving-api-54d89bd5fc-hfvdp\" is forbidden: User \"system:node:ip-10-0-137-60.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-60.ec2.internal' and this object" podUID="bd54ca4e-848b-4c2d-8be0-10214ff2148c" pod="opendatahub/model-serving-api-54d89bd5fc-hfvdp" Jul 24 13:40:24.528942 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:24.528917 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hfvdp"] Jul 24 13:40:24.640358 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:24.640267 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6cvh\" (UniqueName: \"kubernetes.io/projected/bd54ca4e-848b-4c2d-8be0-10214ff2148c-kube-api-access-r6cvh\") pod \"model-serving-api-54d89bd5fc-hfvdp\" (UID: \"bd54ca4e-848b-4c2d-8be0-10214ff2148c\") " pod="opendatahub/model-serving-api-54d89bd5fc-hfvdp" Jul 24 13:40:24.640358 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:24.640327 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd54ca4e-848b-4c2d-8be0-10214ff2148c-tls-certs\") pod \"model-serving-api-54d89bd5fc-hfvdp\" (UID: \"bd54ca4e-848b-4c2d-8be0-10214ff2148c\") " pod="opendatahub/model-serving-api-54d89bd5fc-hfvdp" Jul 24 13:40:24.721357 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:24.721319 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:24.741878 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:24.741399 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6cvh\" (UniqueName: \"kubernetes.io/projected/bd54ca4e-848b-4c2d-8be0-10214ff2148c-kube-api-access-r6cvh\") pod \"model-serving-api-54d89bd5fc-hfvdp\" (UID: \"bd54ca4e-848b-4c2d-8be0-10214ff2148c\") " pod="opendatahub/model-serving-api-54d89bd5fc-hfvdp" Jul 24 13:40:24.741878 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:24.741458 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd54ca4e-848b-4c2d-8be0-10214ff2148c-tls-certs\") pod \"model-serving-api-54d89bd5fc-hfvdp\" (UID: \"bd54ca4e-848b-4c2d-8be0-10214ff2148c\") " pod="opendatahub/model-serving-api-54d89bd5fc-hfvdp" Jul 24 13:40:24.767223 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:24.767184 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6cvh\" (UniqueName: \"kubernetes.io/projected/bd54ca4e-848b-4c2d-8be0-10214ff2148c-kube-api-access-r6cvh\") pod \"model-serving-api-54d89bd5fc-hfvdp\" (UID: \"bd54ca4e-848b-4c2d-8be0-10214ff2148c\") " pod="opendatahub/model-serving-api-54d89bd5fc-hfvdp" Jul 24 13:40:25.116330 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.116297 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b4ln4" event={"ID":"24b46733-ac33-433b-bf13-17bbbbdfc040","Type":"ContainerStarted","Data":"f23907796fbf5ba6d8a71aa06a8c66b822397f09a31d930b8bc6c33ae3cd31dc"} Jul 24 13:40:25.221683 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.221321 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph"] Jul 24 13:40:25.224018 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.223976 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.224309 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.224016 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:25.227292 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.227237 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 24 13:40:25.227292 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.227260 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 24 13:40:25.227458 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.227329 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-k4qsg\"" Jul 24 13:40:25.227458 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.227240 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 13:40:25.227557 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.227486 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 24 13:40:25.243843 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.243768 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph"] Jul 24 13:40:25.318751 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.318665 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:25.354071 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.354033 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srtcm\" (UniqueName: \"kubernetes.io/projected/3ba99b0d-242b-47be-bb17-4483c83a7cf7-kube-api-access-srtcm\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.354255 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.354136 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ba99b0d-242b-47be-bb17-4483c83a7cf7-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.354255 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.354186 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.354255 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.354224 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.354255 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.354249 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.354468 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.354282 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ba99b0d-242b-47be-bb17-4483c83a7cf7-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.354468 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.354311 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.456835 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.455193 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.456835 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.455240 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.456835 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.455278 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ba99b0d-242b-47be-bb17-4483c83a7cf7-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.456835 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.455311 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.456835 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.455373 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srtcm\" (UniqueName: \"kubernetes.io/projected/3ba99b0d-242b-47be-bb17-4483c83a7cf7-kube-api-access-srtcm\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.456835 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.455450 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ba99b0d-242b-47be-bb17-4483c83a7cf7-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.456835 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.455496 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.457636 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.456908 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.462662 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.462609 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ba99b0d-242b-47be-bb17-4483c83a7cf7-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.463187 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.463146 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.465359 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.465317 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ba99b0d-242b-47be-bb17-4483c83a7cf7-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.467562 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.467525 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.471193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.471149 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.475600 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.475557 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srtcm\" (UniqueName: \"kubernetes.io/projected/3ba99b0d-242b-47be-bb17-4483c83a7cf7-kube-api-access-srtcm\") pod \"istiod-openshift-gateway-7b7995d5cd-46bph\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.537495 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.537452 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:25.547837 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.547812 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 24 13:40:25.556664 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.556612 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd54ca4e-848b-4c2d-8be0-10214ff2148c-tls-certs\") pod \"model-serving-api-54d89bd5fc-hfvdp\" (UID: \"bd54ca4e-848b-4c2d-8be0-10214ff2148c\") " pod="opendatahub/model-serving-api-54d89bd5fc-hfvdp" Jul 24 13:40:25.707296 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:25.707213 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hfvdp" Jul 24 13:40:26.677923 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:26.677765 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:27.701204 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:27.701169 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hfvdp"] Jul 24 13:40:27.705352 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:40:27.705307 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd54ca4e_848b_4c2d_8be0_10214ff2148c.slice/crio-63cf3926fb5876f5de5b6f118ab995697aa3b21a43c0a7b29107d153a3fc9840 WatchSource:0}: Error finding container 63cf3926fb5876f5de5b6f118ab995697aa3b21a43c0a7b29107d153a3fc9840: Status 404 returned error can't find the container with id 63cf3926fb5876f5de5b6f118ab995697aa3b21a43c0a7b29107d153a3fc9840 Jul 24 13:40:27.723215 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:27.723188 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph"] Jul 24 13:40:27.725269 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:40:27.725237 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ba99b0d_242b_47be_bb17_4483c83a7cf7.slice/crio-1b235ce92e08487bbbe88f9ae04e547a52c023317d73a451de5bf318f5e2aef5 WatchSource:0}: Error finding container 1b235ce92e08487bbbe88f9ae04e547a52c023317d73a451de5bf318f5e2aef5: Status 404 returned error can't find the container with id 1b235ce92e08487bbbe88f9ae04e547a52c023317d73a451de5bf318f5e2aef5 Jul 24 13:40:28.132469 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:28.132433 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85dc78bf9-5v5rs" event={"ID":"ef5cbd04-8ca5-4930-82a3-a722b0ab36d3","Type":"ContainerStarted","Data":"1a51cec8b37bf778716bfe8d550a8d1046a034aec90d76b21a07b84997bd4261"} Jul 24 13:40:28.133922 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:28.133892 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b4ln4" event={"ID":"24b46733-ac33-433b-bf13-17bbbbdfc040","Type":"ContainerStarted","Data":"7f8b9817016bc7c02b76e651b618a26f05477c17006f96cda110322104b36c93"} Jul 24 13:40:28.134086 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:28.133936 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-b4ln4" Jul 24 13:40:28.135113 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:28.135091 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" event={"ID":"3ba99b0d-242b-47be-bb17-4483c83a7cf7","Type":"ContainerStarted","Data":"1b235ce92e08487bbbe88f9ae04e547a52c023317d73a451de5bf318f5e2aef5"} Jul 24 13:40:28.136113 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:28.136088 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hfvdp" event={"ID":"bd54ca4e-848b-4c2d-8be0-10214ff2148c","Type":"ContainerStarted","Data":"63cf3926fb5876f5de5b6f118ab995697aa3b21a43c0a7b29107d153a3fc9840"} Jul 24 13:40:28.152465 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:28.152400 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-85dc78bf9-5v5rs" podStartSLOduration=1.257776284 podStartE2EDuration="5.1523808s" totalImagesPullingTime="3.894604516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:40:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:40:28.152097533 +0000 UTC m=+370.275479933" watchObservedRunningTime="2026-07-24 13:40:28.1523808 +0000 UTC m=+370.275763200" Jul 24 13:40:28.174772 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:28.173139 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-b4ln4" podStartSLOduration=2.141660301 podStartE2EDuration="5.17311657s" totalImagesPullingTime="3.031456269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:40:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:40:28.172799111 +0000 UTC m=+370.296181522" watchObservedRunningTime="2026-07-24 13:40:28.17311657 +0000 UTC m=+370.296498969" Jul 24 13:40:30.152641 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:30.152599 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hfvdp" event={"ID":"bd54ca4e-848b-4c2d-8be0-10214ff2148c","Type":"ContainerStarted","Data":"8688aaf262dcc773c69aed14d423464e2781dc8b384c02396f7575fc32ba5d3c"} Jul 24 13:40:30.153153 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:30.152715 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-hfvdp" Jul 24 13:40:30.172669 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:30.172615 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-hfvdp" podStartSLOduration=4.273399386 podStartE2EDuration="6.172594519s" totalImagesPullingTime="1.899195133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:40:30.172148086 +0000 UTC m=+372.295530482" watchObservedRunningTime="2026-07-24 13:40:30.172594519 +0000 UTC m=+372.295976917" Jul 24 13:40:30.701793 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:30.701753 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:40:30.701888 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:30.701823 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:40:31.159750 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:31.159716 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" event={"ID":"3ba99b0d-242b-47be-bb17-4483c83a7cf7","Type":"ContainerStarted","Data":"450d5b9c33d3a11eeddbb0f1d708a1ad5b661d15d2bd646f379b384ed2b199d4"} Jul 24 13:40:31.160204 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:31.160062 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:31.161529 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:31.161506 2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-46bph container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 13:40:31.161637 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:31.161550 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" podUID="3ba99b0d-242b-47be-bb17-4483c83a7cf7" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 13:40:31.394134 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:31.394098 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:31.490603 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:31.490485 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:32.166306 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.166262 2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-46bph container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 13:40:32.166680 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.166317 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" podUID="3ba99b0d-242b-47be-bb17-4483c83a7cf7" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 13:40:32.402059 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.402003 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" podStartSLOduration=4.427862363 podStartE2EDuration="7.401966043s" totalImagesPullingTime="2.97410368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:40:31.184021215 +0000 UTC m=+373.307403612" watchObservedRunningTime="2026-07-24 13:40:32.401966043 +0000 UTC m=+374.525348442" Jul 24 13:40:32.402940 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.402920 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw"] Jul 24 13:40:32.406228 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.406208 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.409014 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.408972 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-nt6q6\"" Jul 24 13:40:32.426534 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.426435 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw"] Jul 24 13:40:32.441689 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.441659 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:32.451167 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.451139 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9"] Jul 24 13:40:32.453552 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.453529 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.480589 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.480557 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9"] Jul 24 13:40:32.526638 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.526601 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52323b36-8263-4bff-9712-b2b903cd4204-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.526832 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.526704 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52323b36-8263-4bff-9712-b2b903cd4204-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.526832 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.526731 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52323b36-8263-4bff-9712-b2b903cd4204-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.526832 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.526757 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52323b36-8263-4bff-9712-b2b903cd4204-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.526832 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.526793 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52323b36-8263-4bff-9712-b2b903cd4204-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.526832 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.526820 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd792\" (UniqueName: \"kubernetes.io/projected/52323b36-8263-4bff-9712-b2b903cd4204-kube-api-access-kd792\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.527131 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.526856 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52323b36-8263-4bff-9712-b2b903cd4204-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.527131 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.526914 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52323b36-8263-4bff-9712-b2b903cd4204-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.527131 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.527075 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52323b36-8263-4bff-9712-b2b903cd4204-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.619357 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.619317 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:32.627679 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.627650 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52323b36-8263-4bff-9712-b2b903cd4204-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.627679 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.627682 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52323b36-8263-4bff-9712-b2b903cd4204-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.627894 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.627700 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52323b36-8263-4bff-9712-b2b903cd4204-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.627894 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.627720 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d685f64b-4b49-4930-b50a-b45df2488ba2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.627975 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.627918 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52323b36-8263-4bff-9712-b2b903cd4204-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.627975 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.627963 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd792\" (UniqueName: \"kubernetes.io/projected/52323b36-8263-4bff-9712-b2b903cd4204-kube-api-access-kd792\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.628101 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.628030 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52323b36-8263-4bff-9712-b2b903cd4204-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.628101 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.628060 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crzs6\" (UniqueName: \"kubernetes.io/projected/d685f64b-4b49-4930-b50a-b45df2488ba2-kube-api-access-crzs6\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.628101 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.628085 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52323b36-8263-4bff-9712-b2b903cd4204-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.628249 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.628101 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52323b36-8263-4bff-9712-b2b903cd4204-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.628249 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.628154 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52323b36-8263-4bff-9712-b2b903cd4204-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.628249 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.628172 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d685f64b-4b49-4930-b50a-b45df2488ba2-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.628249 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.628213 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52323b36-8263-4bff-9712-b2b903cd4204-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.628249 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.628238 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d685f64b-4b49-4930-b50a-b45df2488ba2-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.628473 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.628289 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d685f64b-4b49-4930-b50a-b45df2488ba2-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.628473 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.628326 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52323b36-8263-4bff-9712-b2b903cd4204-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.628473 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.628327 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52323b36-8263-4bff-9712-b2b903cd4204-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.628473 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.628379 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d685f64b-4b49-4930-b50a-b45df2488ba2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.628473 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.628409 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d685f64b-4b49-4930-b50a-b45df2488ba2-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.628668 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.628530 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52323b36-8263-4bff-9712-b2b903cd4204-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.628935 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.628914 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52323b36-8263-4bff-9712-b2b903cd4204-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.630586 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.630559 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52323b36-8263-4bff-9712-b2b903cd4204-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.630797 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.630777 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52323b36-8263-4bff-9712-b2b903cd4204-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.644107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.644085 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52323b36-8263-4bff-9712-b2b903cd4204-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.649692 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.649671 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd792\" (UniqueName: \"kubernetes.io/projected/52323b36-8263-4bff-9712-b2b903cd4204-kube-api-access-kd792\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw\" (UID: \"52323b36-8263-4bff-9712-b2b903cd4204\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.716749 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.716659 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:32.729689 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.729658 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d685f64b-4b49-4930-b50a-b45df2488ba2-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.729804 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.729701 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d685f64b-4b49-4930-b50a-b45df2488ba2-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.729804 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.729728 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d685f64b-4b49-4930-b50a-b45df2488ba2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.729804 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.729745 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d685f64b-4b49-4930-b50a-b45df2488ba2-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.729804 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.729783 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d685f64b-4b49-4930-b50a-b45df2488ba2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.730060 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.729810 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crzs6\" (UniqueName: \"kubernetes.io/projected/d685f64b-4b49-4930-b50a-b45df2488ba2-kube-api-access-crzs6\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.730060 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.729848 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d685f64b-4b49-4930-b50a-b45df2488ba2-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.730985 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.730963 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d685f64b-4b49-4930-b50a-b45df2488ba2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.732785 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.732764 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d685f64b-4b49-4930-b50a-b45df2488ba2-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.732965 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.732814 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d685f64b-4b49-4930-b50a-b45df2488ba2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.732965 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.732847 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d685f64b-4b49-4930-b50a-b45df2488ba2-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.733202 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.733183 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d685f64b-4b49-4930-b50a-b45df2488ba2-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.744852 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.744826 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crzs6\" (UniqueName: \"kubernetes.io/projected/d685f64b-4b49-4930-b50a-b45df2488ba2-kube-api-access-crzs6\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.745414 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.745390 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d685f64b-4b49-4930-b50a-b45df2488ba2-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-kxsk9\" (UID: \"d685f64b-4b49-4930-b50a-b45df2488ba2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.763241 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.763207 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:32.792506 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.792432 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:32.889485 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.889291 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw"] Jul 24 13:40:32.892584 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:40:32.892550 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52323b36_8263_4bff_9712_b2b903cd4204.slice/crio-81f8a2ba1f1d660b14aa0dc5eea80e5c3fc22cee1b2f46fc1d4c38db176ded45 WatchSource:0}: Error finding container 81f8a2ba1f1d660b14aa0dc5eea80e5c3fc22cee1b2f46fc1d4c38db176ded45: Status 404 returned error can't find the container with id 81f8a2ba1f1d660b14aa0dc5eea80e5c3fc22cee1b2f46fc1d4c38db176ded45 Jul 24 13:40:32.935225 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.935199 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9"] Jul 24 13:40:32.937482 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:40:32.937441 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd685f64b_4b49_4930_b50a_b45df2488ba2.slice/crio-2fb7761042b5d71fb3bcfcd09b52a570aa5cfb18c5a21a05b5298312f8250c60 WatchSource:0}: Error finding container 2fb7761042b5d71fb3bcfcd09b52a570aa5cfb18c5a21a05b5298312f8250c60: Status 404 returned error can't find the container with id 2fb7761042b5d71fb3bcfcd09b52a570aa5cfb18c5a21a05b5298312f8250c60 Jul 24 13:40:32.940686 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.940645 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:40:32.940776 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:32.940716 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:40:33.172075 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:33.172035 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" event={"ID":"d685f64b-4b49-4930-b50a-b45df2488ba2","Type":"ContainerStarted","Data":"9a8542105e470b041347a4a75ee4989064e82e34244837ae3fe3b7583491ca79"} Jul 24 13:40:33.172509 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:33.172088 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" event={"ID":"d685f64b-4b49-4930-b50a-b45df2488ba2","Type":"ContainerStarted","Data":"2fb7761042b5d71fb3bcfcd09b52a570aa5cfb18c5a21a05b5298312f8250c60"} Jul 24 13:40:33.172509 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:33.172145 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:33.173553 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:33.173525 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" event={"ID":"52323b36-8263-4bff-9712-b2b903cd4204","Type":"ContainerStarted","Data":"81f8a2ba1f1d660b14aa0dc5eea80e5c3fc22cee1b2f46fc1d4c38db176ded45"} Jul 24 13:40:33.211054 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:33.211005 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" podStartSLOduration=1.21096935 podStartE2EDuration="1.21096935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:40:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:40:33.209779082 +0000 UTC m=+375.333161482" watchObservedRunningTime="2026-07-24 13:40:33.21096935 +0000 UTC m=+375.334351749" Jul 24 13:40:34.181592 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.181551 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kxsk9" Jul 24 13:40:34.262393 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.262359 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph"] Jul 24 13:40:34.262642 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.262613 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" podUID="3ba99b0d-242b-47be-bb17-4483c83a7cf7" containerName="discovery" containerID="cri-o://450d5b9c33d3a11eeddbb0f1d708a1ad5b661d15d2bd646f379b384ed2b199d4" gracePeriod=30 Jul 24 13:40:34.263520 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.263340 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:34.554295 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.553144 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:34.651456 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.651416 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ba99b0d-242b-47be-bb17-4483c83a7cf7-cacerts\") pod \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " Jul 24 13:40:34.651625 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.651498 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-csr-dns-cert\") pod \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " Jul 24 13:40:34.651625 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.651579 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-token\") pod \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " Jul 24 13:40:34.651625 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.651611 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-csr-ca-configmap\") pod \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " Jul 24 13:40:34.651811 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.651640 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-kubeconfig\") pod \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " Jul 24 13:40:34.651811 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.651665 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srtcm\" (UniqueName: \"kubernetes.io/projected/3ba99b0d-242b-47be-bb17-4483c83a7cf7-kube-api-access-srtcm\") pod \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " Jul 24 13:40:34.651811 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.651689 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ba99b0d-242b-47be-bb17-4483c83a7cf7-local-certs\") pod \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\" (UID: \"3ba99b0d-242b-47be-bb17-4483c83a7cf7\") " Jul 24 13:40:34.652274 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.652237 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "3ba99b0d-242b-47be-bb17-4483c83a7cf7" (UID: "3ba99b0d-242b-47be-bb17-4483c83a7cf7"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:40:34.655846 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.655800 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "3ba99b0d-242b-47be-bb17-4483c83a7cf7" (UID: "3ba99b0d-242b-47be-bb17-4483c83a7cf7"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:40:34.655846 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.655822 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ba99b0d-242b-47be-bb17-4483c83a7cf7-kube-api-access-srtcm" (OuterVolumeSpecName: "kube-api-access-srtcm") pod "3ba99b0d-242b-47be-bb17-4483c83a7cf7" (UID: "3ba99b0d-242b-47be-bb17-4483c83a7cf7"). InnerVolumeSpecName "kube-api-access-srtcm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:40:34.656072 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.655946 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "3ba99b0d-242b-47be-bb17-4483c83a7cf7" (UID: "3ba99b0d-242b-47be-bb17-4483c83a7cf7"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:40:34.657126 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.657100 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ba99b0d-242b-47be-bb17-4483c83a7cf7-local-certs" (OuterVolumeSpecName: "local-certs") pod "3ba99b0d-242b-47be-bb17-4483c83a7cf7" (UID: "3ba99b0d-242b-47be-bb17-4483c83a7cf7"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:40:34.657466 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.657441 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ba99b0d-242b-47be-bb17-4483c83a7cf7-cacerts" (OuterVolumeSpecName: "cacerts") pod "3ba99b0d-242b-47be-bb17-4483c83a7cf7" (UID: "3ba99b0d-242b-47be-bb17-4483c83a7cf7"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:40:34.657580 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.657468 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-token" (OuterVolumeSpecName: "istio-token") pod "3ba99b0d-242b-47be-bb17-4483c83a7cf7" (UID: "3ba99b0d-242b-47be-bb17-4483c83a7cf7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:40:34.752554 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.752475 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-token\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:40:34.752554 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.752513 2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-csr-ca-configmap\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:40:34.752554 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.752529 2579 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-kubeconfig\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:40:34.752554 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.752543 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srtcm\" (UniqueName: \"kubernetes.io/projected/3ba99b0d-242b-47be-bb17-4483c83a7cf7-kube-api-access-srtcm\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:40:34.752554 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.752556 2579 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ba99b0d-242b-47be-bb17-4483c83a7cf7-local-certs\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:40:34.752825 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.752569 2579 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ba99b0d-242b-47be-bb17-4483c83a7cf7-cacerts\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:40:34.752825 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:34.752582 2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ba99b0d-242b-47be-bb17-4483c83a7cf7-istio-csr-dns-cert\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:40:35.185497 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.185448 2579 generic.go:356] "Generic (PLEG): container finished" podID="3ba99b0d-242b-47be-bb17-4483c83a7cf7" containerID="450d5b9c33d3a11eeddbb0f1d708a1ad5b661d15d2bd646f379b384ed2b199d4" exitCode=0 Jul 24 13:40:35.185497 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.185499 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" Jul 24 13:40:35.186065 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.185513 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" event={"ID":"3ba99b0d-242b-47be-bb17-4483c83a7cf7","Type":"ContainerDied","Data":"450d5b9c33d3a11eeddbb0f1d708a1ad5b661d15d2bd646f379b384ed2b199d4"} Jul 24 13:40:35.186065 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.185574 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph" event={"ID":"3ba99b0d-242b-47be-bb17-4483c83a7cf7","Type":"ContainerDied","Data":"1b235ce92e08487bbbe88f9ae04e547a52c023317d73a451de5bf318f5e2aef5"} Jul 24 13:40:35.186065 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.185604 2579 scope.go:117] "RemoveContainer" containerID="450d5b9c33d3a11eeddbb0f1d708a1ad5b661d15d2bd646f379b384ed2b199d4" Jul 24 13:40:35.236141 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.236080 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph"] Jul 24 13:40:35.240695 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.240659 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-46bph"] Jul 24 13:40:35.442715 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.442631 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:35.523081 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.523048 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-h49g2"] Jul 24 13:40:35.523471 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.523448 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ba99b0d-242b-47be-bb17-4483c83a7cf7" containerName="discovery" Jul 24 13:40:35.523471 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.523465 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ba99b0d-242b-47be-bb17-4483c83a7cf7" containerName="discovery" Jul 24 13:40:35.523596 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.523520 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ba99b0d-242b-47be-bb17-4483c83a7cf7" containerName="discovery" Jul 24 13:40:35.529518 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.529490 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h49g2" Jul 24 13:40:35.532852 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.532827 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 24 13:40:35.533018 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.532827 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-r79gx\"" Jul 24 13:40:35.536327 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.536305 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4rfzs"] Jul 24 13:40:35.541104 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.541082 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rfzs" Jul 24 13:40:35.545983 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.545958 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-4756p\"" Jul 24 13:40:35.546263 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.546242 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 24 13:40:35.548250 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.548230 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-h49g2"] Jul 24 13:40:35.558113 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.558090 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4rfzs"] Jul 24 13:40:35.564458 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.564416 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27qvx\" (UniqueName: \"kubernetes.io/projected/e088b46b-0f4e-4740-a42c-bcd1d04be8c8-kube-api-access-27qvx\") pod \"kserve-controller-manager-d6f4b65b8-h49g2\" (UID: \"e088b46b-0f4e-4740-a42c-bcd1d04be8c8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h49g2" Jul 24 13:40:35.564572 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.564534 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4424e0d0-b6e6-44a0-82e9-a098570c212a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4rfzs\" (UID: \"4424e0d0-b6e6-44a0-82e9-a098570c212a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rfzs" Jul 24 13:40:35.564572 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.564557 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgjd7\" (UniqueName: \"kubernetes.io/projected/4424e0d0-b6e6-44a0-82e9-a098570c212a-kube-api-access-vgjd7\") pod \"llmisvc-controller-manager-777c6bd7d5-4rfzs\" (UID: \"4424e0d0-b6e6-44a0-82e9-a098570c212a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rfzs" Jul 24 13:40:35.564684 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.564650 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e088b46b-0f4e-4740-a42c-bcd1d04be8c8-cert\") pod \"kserve-controller-manager-d6f4b65b8-h49g2\" (UID: \"e088b46b-0f4e-4740-a42c-bcd1d04be8c8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h49g2" Jul 24 13:40:35.581664 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.581630 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:35.657977 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.657936 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:35.665113 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.665071 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27qvx\" (UniqueName: \"kubernetes.io/projected/e088b46b-0f4e-4740-a42c-bcd1d04be8c8-kube-api-access-27qvx\") pod \"kserve-controller-manager-d6f4b65b8-h49g2\" (UID: \"e088b46b-0f4e-4740-a42c-bcd1d04be8c8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h49g2" Jul 24 13:40:35.665301 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.665153 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4424e0d0-b6e6-44a0-82e9-a098570c212a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4rfzs\" (UID: \"4424e0d0-b6e6-44a0-82e9-a098570c212a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rfzs" Jul 24 13:40:35.665301 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.665178 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgjd7\" (UniqueName: \"kubernetes.io/projected/4424e0d0-b6e6-44a0-82e9-a098570c212a-kube-api-access-vgjd7\") pod \"llmisvc-controller-manager-777c6bd7d5-4rfzs\" (UID: \"4424e0d0-b6e6-44a0-82e9-a098570c212a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rfzs" Jul 24 13:40:35.665301 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.665260 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e088b46b-0f4e-4740-a42c-bcd1d04be8c8-cert\") pod \"kserve-controller-manager-d6f4b65b8-h49g2\" (UID: \"e088b46b-0f4e-4740-a42c-bcd1d04be8c8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h49g2" Jul 24 13:40:35.665301 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:40:35.665285 2579 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 24 13:40:35.665541 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:40:35.665352 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4424e0d0-b6e6-44a0-82e9-a098570c212a-cert podName:4424e0d0-b6e6-44a0-82e9-a098570c212a nodeName:}" failed. No retries permitted until 2026-07-24 13:40:36.165332132 +0000 UTC m=+378.288714528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4424e0d0-b6e6-44a0-82e9-a098570c212a-cert") pod "llmisvc-controller-manager-777c6bd7d5-4rfzs" (UID: "4424e0d0-b6e6-44a0-82e9-a098570c212a") : secret "llmisvc-webhook-server-cert" not found Jul 24 13:40:35.668093 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.668071 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e088b46b-0f4e-4740-a42c-bcd1d04be8c8-cert\") pod \"kserve-controller-manager-d6f4b65b8-h49g2\" (UID: \"e088b46b-0f4e-4740-a42c-bcd1d04be8c8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h49g2" Jul 24 13:40:35.683429 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.683403 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27qvx\" (UniqueName: \"kubernetes.io/projected/e088b46b-0f4e-4740-a42c-bcd1d04be8c8-kube-api-access-27qvx\") pod \"kserve-controller-manager-d6f4b65b8-h49g2\" (UID: \"e088b46b-0f4e-4740-a42c-bcd1d04be8c8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h49g2" Jul 24 13:40:35.683694 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.683672 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgjd7\" (UniqueName: \"kubernetes.io/projected/4424e0d0-b6e6-44a0-82e9-a098570c212a-kube-api-access-vgjd7\") pod \"llmisvc-controller-manager-777c6bd7d5-4rfzs\" (UID: \"4424e0d0-b6e6-44a0-82e9-a098570c212a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rfzs" Jul 24 13:40:35.844712 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.844671 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h49g2" Jul 24 13:40:35.986395 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.986370 2579 scope.go:117] "RemoveContainer" containerID="450d5b9c33d3a11eeddbb0f1d708a1ad5b661d15d2bd646f379b384ed2b199d4" Jul 24 13:40:35.986777 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:40:35.986753 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"450d5b9c33d3a11eeddbb0f1d708a1ad5b661d15d2bd646f379b384ed2b199d4\": container with ID starting with 450d5b9c33d3a11eeddbb0f1d708a1ad5b661d15d2bd646f379b384ed2b199d4 not found: ID does not exist" containerID="450d5b9c33d3a11eeddbb0f1d708a1ad5b661d15d2bd646f379b384ed2b199d4" Jul 24 13:40:35.986843 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:35.986786 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"450d5b9c33d3a11eeddbb0f1d708a1ad5b661d15d2bd646f379b384ed2b199d4"} err="failed to get container status \"450d5b9c33d3a11eeddbb0f1d708a1ad5b661d15d2bd646f379b384ed2b199d4\": rpc error: code = NotFound desc = could not find container \"450d5b9c33d3a11eeddbb0f1d708a1ad5b661d15d2bd646f379b384ed2b199d4\": container with ID starting with 450d5b9c33d3a11eeddbb0f1d708a1ad5b661d15d2bd646f379b384ed2b199d4 not found: ID does not exist" Jul 24 13:40:36.036582 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:36.036540 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:40:36.036697 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:36.036627 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:40:36.036697 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:36.036660 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:40:36.041099 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:36.041034 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:36.128818 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:36.128790 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-h49g2"] Jul 24 13:40:36.131875 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:40:36.131840 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode088b46b_0f4e_4740_a42c_bcd1d04be8c8.slice/crio-149bb69a3e85b3de2c7b5f15de1345537b052b64dea25af81f7038bffa5b2cf5 WatchSource:0}: Error finding container 149bb69a3e85b3de2c7b5f15de1345537b052b64dea25af81f7038bffa5b2cf5: Status 404 returned error can't find the container with id 149bb69a3e85b3de2c7b5f15de1345537b052b64dea25af81f7038bffa5b2cf5 Jul 24 13:40:36.169494 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:36.169463 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4424e0d0-b6e6-44a0-82e9-a098570c212a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4rfzs\" (UID: \"4424e0d0-b6e6-44a0-82e9-a098570c212a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rfzs" Jul 24 13:40:36.172105 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:36.172071 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4424e0d0-b6e6-44a0-82e9-a098570c212a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4rfzs\" (UID: \"4424e0d0-b6e6-44a0-82e9-a098570c212a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rfzs" Jul 24 13:40:36.191263 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:36.191234 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h49g2" event={"ID":"e088b46b-0f4e-4740-a42c-bcd1d04be8c8","Type":"ContainerStarted","Data":"149bb69a3e85b3de2c7b5f15de1345537b052b64dea25af81f7038bffa5b2cf5"} Jul 24 13:40:36.193780 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:36.193747 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" event={"ID":"52323b36-8263-4bff-9712-b2b903cd4204","Type":"ContainerStarted","Data":"0cba5092e530f47a76e3819e2502fbcbd0599da9ae24f57c1731979316dde56a"} Jul 24 13:40:36.228073 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:36.228022 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" podStartSLOduration=1.087227973 podStartE2EDuration="4.227984922s" totalImagesPullingTime="3.140756949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:40:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:40:36.226099853 +0000 UTC m=+378.349482250" watchObservedRunningTime="2026-07-24 13:40:36.227984922 +0000 UTC m=+378.351367320" Jul 24 13:40:36.419496 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:36.419413 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ba99b0d-242b-47be-bb17-4483c83a7cf7" path="/var/lib/kubelet/pods/3ba99b0d-242b-47be-bb17-4483c83a7cf7/volumes" Jul 24 13:40:36.457305 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:36.457278 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rfzs" Jul 24 13:40:36.627497 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:36.627472 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4rfzs"] Jul 24 13:40:36.627852 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:40:36.627819 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4424e0d0_b6e6_44a0_82e9_a098570c212a.slice/crio-b117bfd2f2125334da497fcc51ec8ae4905b6065787b07c5ae8508a46135031d WatchSource:0}: Error finding container b117bfd2f2125334da497fcc51ec8ae4905b6065787b07c5ae8508a46135031d: Status 404 returned error can't find the container with id b117bfd2f2125334da497fcc51ec8ae4905b6065787b07c5ae8508a46135031d Jul 24 13:40:36.717417 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:36.717336 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:36.845329 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:36.845295 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:36.934728 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:36.934692 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:37.045644 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:37.045556 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:37.200807 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:37.200771 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rfzs" event={"ID":"4424e0d0-b6e6-44a0-82e9-a098570c212a","Type":"ContainerStarted","Data":"b117bfd2f2125334da497fcc51ec8ae4905b6065787b07c5ae8508a46135031d"} Jul 24 13:40:37.722911 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:37.722882 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:38.208132 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:38.208096 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:38.209231 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:38.209204 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw" Jul 24 13:40:39.146662 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:39.146624 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-b4ln4" Jul 24 13:40:40.227349 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:40.227316 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h49g2" event={"ID":"e088b46b-0f4e-4740-a42c-bcd1d04be8c8","Type":"ContainerStarted","Data":"3b13fec22bea61d9d14c41dccda04070313fac687ab67889a698fc2d684f80f5"} Jul 24 13:40:40.227778 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:40.227426 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h49g2" Jul 24 13:40:40.228874 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:40.228848 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rfzs" event={"ID":"4424e0d0-b6e6-44a0-82e9-a098570c212a","Type":"ContainerStarted","Data":"21da0cc366cba8f37e69be301ddef975c8e88f5a969ac40c1b888ec0ebbd3ffd"} Jul 24 13:40:40.229101 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:40.229088 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rfzs" Jul 24 13:40:40.251394 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:40.251350 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h49g2" podStartSLOduration=2.634463137 podStartE2EDuration="5.251333718s" totalImagesPullingTime="2.616870581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:40:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:40:40.250620013 +0000 UTC m=+382.374002412" watchObservedRunningTime="2026-07-24 13:40:40.251333718 +0000 UTC m=+382.374716115" Jul 24 13:40:40.270898 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:40.270846 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rfzs" podStartSLOduration=2.389054702 podStartE2EDuration="5.270827194s" totalImagesPullingTime="2.881772492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:40:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:40:40.270628173 +0000 UTC m=+382.394010575" watchObservedRunningTime="2026-07-24 13:40:40.270827194 +0000 UTC m=+382.394209593" Jul 24 13:40:41.163721 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:41.163694 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-hfvdp" Jul 24 13:40:53.965063 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:53.965019 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:54.069053 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:40:54.069017 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:11.236325 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:11.236285 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rfzs" Jul 24 13:41:11.239395 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:11.239374 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h49g2" Jul 24 13:41:18.023349 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:18.023315 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:24.095858 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:24.095818 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:24.236262 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:24.236226 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:24.334102 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:24.334061 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:24.962413 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:24.962369 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:25.182021 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:25.181954 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:40.566756 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.566678 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm"] Jul 24 13:41:40.569477 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.569462 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm" Jul 24 13:41:40.572563 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.572541 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 24 13:41:40.574040 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.574021 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wbnz6\"" Jul 24 13:41:40.574040 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.574035 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 24 13:41:40.581760 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.581739 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm"] Jul 24 13:41:40.651675 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.651647 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a16ceb0-9322-4f1e-b832-434602ebaf83-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm\" (UID: \"2a16ceb0-9322-4f1e-b832-434602ebaf83\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm" Jul 24 13:41:40.651842 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.651680 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fbk5\" (UniqueName: \"kubernetes.io/projected/2a16ceb0-9322-4f1e-b832-434602ebaf83-kube-api-access-5fbk5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm\" (UID: \"2a16ceb0-9322-4f1e-b832-434602ebaf83\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm" Jul 24 13:41:40.651842 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.651703 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a16ceb0-9322-4f1e-b832-434602ebaf83-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm\" (UID: \"2a16ceb0-9322-4f1e-b832-434602ebaf83\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm" Jul 24 13:41:40.752658 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.752627 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a16ceb0-9322-4f1e-b832-434602ebaf83-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm\" (UID: \"2a16ceb0-9322-4f1e-b832-434602ebaf83\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm" Jul 24 13:41:40.752658 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.752661 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fbk5\" (UniqueName: \"kubernetes.io/projected/2a16ceb0-9322-4f1e-b832-434602ebaf83-kube-api-access-5fbk5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm\" (UID: \"2a16ceb0-9322-4f1e-b832-434602ebaf83\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm" Jul 24 13:41:40.752901 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.752683 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a16ceb0-9322-4f1e-b832-434602ebaf83-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm\" (UID: \"2a16ceb0-9322-4f1e-b832-434602ebaf83\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm" Jul 24 13:41:40.753049 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.752984 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a16ceb0-9322-4f1e-b832-434602ebaf83-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm\" (UID: \"2a16ceb0-9322-4f1e-b832-434602ebaf83\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm" Jul 24 13:41:40.753120 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.753049 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a16ceb0-9322-4f1e-b832-434602ebaf83-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm\" (UID: \"2a16ceb0-9322-4f1e-b832-434602ebaf83\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm" Jul 24 13:41:40.764844 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.764812 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fbk5\" (UniqueName: \"kubernetes.io/projected/2a16ceb0-9322-4f1e-b832-434602ebaf83-kube-api-access-5fbk5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm\" (UID: \"2a16ceb0-9322-4f1e-b832-434602ebaf83\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm" Jul 24 13:41:40.878299 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.878214 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm" Jul 24 13:41:40.951666 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.951636 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf"] Jul 24 13:41:40.954643 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.954623 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf" Jul 24 13:41:40.964247 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:40.964203 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf"] Jul 24 13:41:41.027406 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.027381 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm"] Jul 24 13:41:41.029631 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:41:41.029602 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a16ceb0_9322_4f1e_b832_434602ebaf83.slice/crio-f1d6ed54351e7255ec6260d43bf3105435a86c3229e7c30be37720dce269c86c WatchSource:0}: Error finding container f1d6ed54351e7255ec6260d43bf3105435a86c3229e7c30be37720dce269c86c: Status 404 returned error can't find the container with id f1d6ed54351e7255ec6260d43bf3105435a86c3229e7c30be37720dce269c86c Jul 24 13:41:41.055741 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.055709 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3174010-56ea-472b-b6c0-f74cb8bcde98-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf\" (UID: \"d3174010-56ea-472b-b6c0-f74cb8bcde98\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf" Jul 24 13:41:41.055885 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.055769 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3174010-56ea-472b-b6c0-f74cb8bcde98-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf\" (UID: \"d3174010-56ea-472b-b6c0-f74cb8bcde98\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf" Jul 24 13:41:41.055956 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.055887 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nbv5\" (UniqueName: \"kubernetes.io/projected/d3174010-56ea-472b-b6c0-f74cb8bcde98-kube-api-access-5nbv5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf\" (UID: \"d3174010-56ea-472b-b6c0-f74cb8bcde98\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf" Jul 24 13:41:41.157394 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.157358 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3174010-56ea-472b-b6c0-f74cb8bcde98-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf\" (UID: \"d3174010-56ea-472b-b6c0-f74cb8bcde98\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf" Jul 24 13:41:41.157706 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.157416 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3174010-56ea-472b-b6c0-f74cb8bcde98-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf\" (UID: \"d3174010-56ea-472b-b6c0-f74cb8bcde98\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf" Jul 24 13:41:41.157706 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.157462 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nbv5\" (UniqueName: \"kubernetes.io/projected/d3174010-56ea-472b-b6c0-f74cb8bcde98-kube-api-access-5nbv5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf\" (UID: \"d3174010-56ea-472b-b6c0-f74cb8bcde98\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf" Jul 24 13:41:41.157797 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.157729 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3174010-56ea-472b-b6c0-f74cb8bcde98-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf\" (UID: \"d3174010-56ea-472b-b6c0-f74cb8bcde98\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf" Jul 24 13:41:41.157797 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.157760 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3174010-56ea-472b-b6c0-f74cb8bcde98-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf\" (UID: \"d3174010-56ea-472b-b6c0-f74cb8bcde98\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf" Jul 24 13:41:41.166703 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.166682 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nbv5\" (UniqueName: \"kubernetes.io/projected/d3174010-56ea-472b-b6c0-f74cb8bcde98-kube-api-access-5nbv5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf\" (UID: \"d3174010-56ea-472b-b6c0-f74cb8bcde98\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf" Jul 24 13:41:41.266770 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.266736 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf" Jul 24 13:41:41.356746 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.356713 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf"] Jul 24 13:41:41.364198 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.364173 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf" Jul 24 13:41:41.372691 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.372338 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf"] Jul 24 13:41:41.412767 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.412729 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf"] Jul 24 13:41:41.414943 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:41:41.414920 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3174010_56ea_472b_b6c0_f74cb8bcde98.slice/crio-47277d40803054338b7f13f603c3cbae50deb119a071da4502c0772a2bdf19b5 WatchSource:0}: Error finding container 47277d40803054338b7f13f603c3cbae50deb119a071da4502c0772a2bdf19b5: Status 404 returned error can't find the container with id 47277d40803054338b7f13f603c3cbae50deb119a071da4502c0772a2bdf19b5 Jul 24 13:41:41.459594 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.459571 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thzsj\" (UniqueName: \"kubernetes.io/projected/10be9861-183d-4216-8429-8a7176d40441-kube-api-access-thzsj\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf\" (UID: \"10be9861-183d-4216-8429-8a7176d40441\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf" Jul 24 13:41:41.459687 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.459635 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10be9861-183d-4216-8429-8a7176d40441-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf\" (UID: \"10be9861-183d-4216-8429-8a7176d40441\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf" Jul 24 13:41:41.459771 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.459754 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10be9861-183d-4216-8429-8a7176d40441-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf\" (UID: \"10be9861-183d-4216-8429-8a7176d40441\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf" Jul 24 13:41:41.523068 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.523036 2579 generic.go:356] "Generic (PLEG): container finished" podID="2a16ceb0-9322-4f1e-b832-434602ebaf83" containerID="8cb77f1bba806f2038e93cdd93e6adc019baa6ed9d2343955409ff3152769570" exitCode=0 Jul 24 13:41:41.523204 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.523105 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm" event={"ID":"2a16ceb0-9322-4f1e-b832-434602ebaf83","Type":"ContainerDied","Data":"8cb77f1bba806f2038e93cdd93e6adc019baa6ed9d2343955409ff3152769570"} Jul 24 13:41:41.523204 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.523163 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm" event={"ID":"2a16ceb0-9322-4f1e-b832-434602ebaf83","Type":"ContainerStarted","Data":"f1d6ed54351e7255ec6260d43bf3105435a86c3229e7c30be37720dce269c86c"} Jul 24 13:41:41.524751 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.524726 2579 generic.go:356] "Generic (PLEG): container finished" podID="d3174010-56ea-472b-b6c0-f74cb8bcde98" containerID="fcca6182619478e1f70a8cf810a0e4199ea33d1c970fa9845b3bd116358bb9e4" exitCode=0 Jul 24 13:41:41.524844 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.524797 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf" event={"ID":"d3174010-56ea-472b-b6c0-f74cb8bcde98","Type":"ContainerDied","Data":"fcca6182619478e1f70a8cf810a0e4199ea33d1c970fa9845b3bd116358bb9e4"} Jul 24 13:41:41.524844 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.524839 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf" event={"ID":"d3174010-56ea-472b-b6c0-f74cb8bcde98","Type":"ContainerStarted","Data":"47277d40803054338b7f13f603c3cbae50deb119a071da4502c0772a2bdf19b5"} Jul 24 13:41:41.560359 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.560295 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10be9861-183d-4216-8429-8a7176d40441-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf\" (UID: \"10be9861-183d-4216-8429-8a7176d40441\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf" Jul 24 13:41:41.560539 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.560407 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thzsj\" (UniqueName: \"kubernetes.io/projected/10be9861-183d-4216-8429-8a7176d40441-kube-api-access-thzsj\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf\" (UID: \"10be9861-183d-4216-8429-8a7176d40441\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf" Jul 24 13:41:41.560539 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.560445 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10be9861-183d-4216-8429-8a7176d40441-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf\" (UID: \"10be9861-183d-4216-8429-8a7176d40441\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf" Jul 24 13:41:41.560706 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.560687 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10be9861-183d-4216-8429-8a7176d40441-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf\" (UID: \"10be9861-183d-4216-8429-8a7176d40441\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf" Jul 24 13:41:41.560777 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.560744 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10be9861-183d-4216-8429-8a7176d40441-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf\" (UID: \"10be9861-183d-4216-8429-8a7176d40441\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf" Jul 24 13:41:41.569838 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.569806 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thzsj\" (UniqueName: \"kubernetes.io/projected/10be9861-183d-4216-8429-8a7176d40441-kube-api-access-thzsj\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf\" (UID: \"10be9861-183d-4216-8429-8a7176d40441\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf" Jul 24 13:41:41.676335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.676256 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf" Jul 24 13:41:41.758841 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.758812 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf"] Jul 24 13:41:41.763837 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.763813 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf" Jul 24 13:41:41.772230 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.772206 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf"] Jul 24 13:41:41.814895 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.814866 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf"] Jul 24 13:41:41.816808 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:41:41.816781 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10be9861_183d_4216_8429_8a7176d40441.slice/crio-fdedc815f20ac8090ecb78e2bf7c3655f0157d7000c6a67d2299a07488e9b761 WatchSource:0}: Error finding container fdedc815f20ac8090ecb78e2bf7c3655f0157d7000c6a67d2299a07488e9b761: Status 404 returned error can't find the container with id fdedc815f20ac8090ecb78e2bf7c3655f0157d7000c6a67d2299a07488e9b761 Jul 24 13:41:41.863047 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.863020 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b9f13076-7dba-4f74-8cbd-7be65860bb40-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf\" (UID: \"b9f13076-7dba-4f74-8cbd-7be65860bb40\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf" Jul 24 13:41:41.863152 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.863104 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b9f13076-7dba-4f74-8cbd-7be65860bb40-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf\" (UID: \"b9f13076-7dba-4f74-8cbd-7be65860bb40\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf" Jul 24 13:41:41.863200 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.863165 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqhwh\" (UniqueName: \"kubernetes.io/projected/b9f13076-7dba-4f74-8cbd-7be65860bb40-kube-api-access-tqhwh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf\" (UID: \"b9f13076-7dba-4f74-8cbd-7be65860bb40\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf" Jul 24 13:41:41.964575 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.964545 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b9f13076-7dba-4f74-8cbd-7be65860bb40-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf\" (UID: \"b9f13076-7dba-4f74-8cbd-7be65860bb40\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf" Jul 24 13:41:41.964710 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.964598 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqhwh\" (UniqueName: \"kubernetes.io/projected/b9f13076-7dba-4f74-8cbd-7be65860bb40-kube-api-access-tqhwh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf\" (UID: \"b9f13076-7dba-4f74-8cbd-7be65860bb40\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf" Jul 24 13:41:41.964710 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.964621 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b9f13076-7dba-4f74-8cbd-7be65860bb40-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf\" (UID: \"b9f13076-7dba-4f74-8cbd-7be65860bb40\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf" Jul 24 13:41:41.964897 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.964877 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b9f13076-7dba-4f74-8cbd-7be65860bb40-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf\" (UID: \"b9f13076-7dba-4f74-8cbd-7be65860bb40\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf" Jul 24 13:41:41.964972 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.964910 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b9f13076-7dba-4f74-8cbd-7be65860bb40-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf\" (UID: \"b9f13076-7dba-4f74-8cbd-7be65860bb40\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf" Jul 24 13:41:41.975765 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:41.975734 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqhwh\" (UniqueName: \"kubernetes.io/projected/b9f13076-7dba-4f74-8cbd-7be65860bb40-kube-api-access-tqhwh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf\" (UID: \"b9f13076-7dba-4f74-8cbd-7be65860bb40\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf" Jul 24 13:41:42.090880 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:42.090848 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf" Jul 24 13:41:42.243098 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:42.242933 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf"] Jul 24 13:41:42.261379 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:41:42.261350 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9f13076_7dba_4f74_8cbd_7be65860bb40.slice/crio-8bc51dc8362e12fcc704e36a402bab57645df06b0314c47db591efa99f3ea131 WatchSource:0}: Error finding container 8bc51dc8362e12fcc704e36a402bab57645df06b0314c47db591efa99f3ea131: Status 404 returned error can't find the container with id 8bc51dc8362e12fcc704e36a402bab57645df06b0314c47db591efa99f3ea131 Jul 24 13:41:42.532075 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:42.531977 2579 generic.go:356] "Generic (PLEG): container finished" podID="10be9861-183d-4216-8429-8a7176d40441" containerID="536038306c9e14e3532b22e1cae76d1c3629c8f0e5d279275787823eded98c92" exitCode=0 Jul 24 13:41:42.532075 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:42.532039 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf" event={"ID":"10be9861-183d-4216-8429-8a7176d40441","Type":"ContainerDied","Data":"536038306c9e14e3532b22e1cae76d1c3629c8f0e5d279275787823eded98c92"} Jul 24 13:41:42.532329 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:42.532098 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf" event={"ID":"10be9861-183d-4216-8429-8a7176d40441","Type":"ContainerStarted","Data":"fdedc815f20ac8090ecb78e2bf7c3655f0157d7000c6a67d2299a07488e9b761"} Jul 24 13:41:42.533822 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:42.533786 2579 generic.go:356] "Generic (PLEG): container finished" podID="b9f13076-7dba-4f74-8cbd-7be65860bb40" containerID="682a72574ef7fec9e099dc940d3968005e449fe84c7c3629ec8f6b0984400881" exitCode=0 Jul 24 13:41:42.533929 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:42.533873 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf" event={"ID":"b9f13076-7dba-4f74-8cbd-7be65860bb40","Type":"ContainerDied","Data":"682a72574ef7fec9e099dc940d3968005e449fe84c7c3629ec8f6b0984400881"} Jul 24 13:41:42.533929 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:42.533919 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf" event={"ID":"b9f13076-7dba-4f74-8cbd-7be65860bb40","Type":"ContainerStarted","Data":"8bc51dc8362e12fcc704e36a402bab57645df06b0314c47db591efa99f3ea131"} Jul 24 13:41:42.536031 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:42.535979 2579 generic.go:356] "Generic (PLEG): container finished" podID="2a16ceb0-9322-4f1e-b832-434602ebaf83" containerID="5936c439ff7931f762253afdb57811d7215a72cbe2d956818971d3e90804524a" exitCode=0 Jul 24 13:41:42.536132 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:42.536030 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm" event={"ID":"2a16ceb0-9322-4f1e-b832-434602ebaf83","Type":"ContainerDied","Data":"5936c439ff7931f762253afdb57811d7215a72cbe2d956818971d3e90804524a"} Jul 24 13:41:43.544896 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:43.544867 2579 generic.go:356] "Generic (PLEG): container finished" podID="b9f13076-7dba-4f74-8cbd-7be65860bb40" containerID="02b5ea64131dfd64bfa3f3ca79a75b76bcc43515fb323d3c1129a43fc823231b" exitCode=0 Jul 24 13:41:43.545337 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:43.544929 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf" event={"ID":"b9f13076-7dba-4f74-8cbd-7be65860bb40","Type":"ContainerDied","Data":"02b5ea64131dfd64bfa3f3ca79a75b76bcc43515fb323d3c1129a43fc823231b"} Jul 24 13:41:43.547228 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:43.547206 2579 generic.go:356] "Generic (PLEG): container finished" podID="2a16ceb0-9322-4f1e-b832-434602ebaf83" containerID="e7ce18969f32f10ed40cc9f61121b002976f63e8af68ebc1ba1f558309d68f74" exitCode=0 Jul 24 13:41:43.547336 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:43.547259 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm" event={"ID":"2a16ceb0-9322-4f1e-b832-434602ebaf83","Type":"ContainerDied","Data":"e7ce18969f32f10ed40cc9f61121b002976f63e8af68ebc1ba1f558309d68f74"} Jul 24 13:41:43.549049 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:43.548984 2579 generic.go:356] "Generic (PLEG): container finished" podID="d3174010-56ea-472b-b6c0-f74cb8bcde98" containerID="3b678399c7bc1791a58a5fc4bbd76d01c4e908ae39070562ba962890d8505b96" exitCode=0 Jul 24 13:41:43.549128 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:43.549029 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf" event={"ID":"d3174010-56ea-472b-b6c0-f74cb8bcde98","Type":"ContainerDied","Data":"3b678399c7bc1791a58a5fc4bbd76d01c4e908ae39070562ba962890d8505b96"} Jul 24 13:41:44.555645 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:44.555601 2579 generic.go:356] "Generic (PLEG): container finished" podID="b9f13076-7dba-4f74-8cbd-7be65860bb40" containerID="2c647af0ed3053c8846d4fc6147bb9a35f9343370b227b1508748bd980bcf5d8" exitCode=0 Jul 24 13:41:44.556068 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:44.555682 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf" event={"ID":"b9f13076-7dba-4f74-8cbd-7be65860bb40","Type":"ContainerDied","Data":"2c647af0ed3053c8846d4fc6147bb9a35f9343370b227b1508748bd980bcf5d8"} Jul 24 13:41:44.557801 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:44.557778 2579 generic.go:356] "Generic (PLEG): container finished" podID="d3174010-56ea-472b-b6c0-f74cb8bcde98" containerID="0cdeb6a2c4f7b3c3378e0822b20b44dd451179ac6c5707566d54a45420515efb" exitCode=0 Jul 24 13:41:44.557926 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:44.557850 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf" event={"ID":"d3174010-56ea-472b-b6c0-f74cb8bcde98","Type":"ContainerDied","Data":"0cdeb6a2c4f7b3c3378e0822b20b44dd451179ac6c5707566d54a45420515efb"} Jul 24 13:41:44.559597 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:44.559576 2579 generic.go:356] "Generic (PLEG): container finished" podID="10be9861-183d-4216-8429-8a7176d40441" containerID="8b53362f97416eaf64f11f4fd79c722b77c0bd900fb54bbfbcbe2409e14f1d33" exitCode=0 Jul 24 13:41:44.559676 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:44.559642 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf" event={"ID":"10be9861-183d-4216-8429-8a7176d40441","Type":"ContainerDied","Data":"8b53362f97416eaf64f11f4fd79c722b77c0bd900fb54bbfbcbe2409e14f1d33"} Jul 24 13:41:44.706308 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:44.706283 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm" Jul 24 13:41:44.789078 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:44.789048 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a16ceb0-9322-4f1e-b832-434602ebaf83-util\") pod \"2a16ceb0-9322-4f1e-b832-434602ebaf83\" (UID: \"2a16ceb0-9322-4f1e-b832-434602ebaf83\") " Jul 24 13:41:44.789269 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:44.789084 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a16ceb0-9322-4f1e-b832-434602ebaf83-bundle\") pod \"2a16ceb0-9322-4f1e-b832-434602ebaf83\" (UID: \"2a16ceb0-9322-4f1e-b832-434602ebaf83\") " Jul 24 13:41:44.789269 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:44.789120 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fbk5\" (UniqueName: \"kubernetes.io/projected/2a16ceb0-9322-4f1e-b832-434602ebaf83-kube-api-access-5fbk5\") pod \"2a16ceb0-9322-4f1e-b832-434602ebaf83\" (UID: \"2a16ceb0-9322-4f1e-b832-434602ebaf83\") " Jul 24 13:41:44.789583 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:44.789558 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a16ceb0-9322-4f1e-b832-434602ebaf83-bundle" (OuterVolumeSpecName: "bundle") pod "2a16ceb0-9322-4f1e-b832-434602ebaf83" (UID: "2a16ceb0-9322-4f1e-b832-434602ebaf83"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:41:44.791459 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:44.791439 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a16ceb0-9322-4f1e-b832-434602ebaf83-kube-api-access-5fbk5" (OuterVolumeSpecName: "kube-api-access-5fbk5") pod "2a16ceb0-9322-4f1e-b832-434602ebaf83" (UID: "2a16ceb0-9322-4f1e-b832-434602ebaf83"). InnerVolumeSpecName "kube-api-access-5fbk5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:41:44.794770 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:44.794731 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a16ceb0-9322-4f1e-b832-434602ebaf83-util" (OuterVolumeSpecName: "util") pod "2a16ceb0-9322-4f1e-b832-434602ebaf83" (UID: "2a16ceb0-9322-4f1e-b832-434602ebaf83"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:41:44.890791 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:44.890753 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fbk5\" (UniqueName: \"kubernetes.io/projected/2a16ceb0-9322-4f1e-b832-434602ebaf83-kube-api-access-5fbk5\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:41:44.890791 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:44.890785 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a16ceb0-9322-4f1e-b832-434602ebaf83-util\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:41:44.890791 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:44.890795 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a16ceb0-9322-4f1e-b832-434602ebaf83-bundle\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:41:45.567317 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.567286 2579 generic.go:356] "Generic (PLEG): container finished" podID="10be9861-183d-4216-8429-8a7176d40441" containerID="dddb54d6de96ed78ff5f0831b97f3e21e94a7745b11a6aac6a05832d486ebd76" exitCode=0 Jul 24 13:41:45.567699 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.567323 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf" event={"ID":"10be9861-183d-4216-8429-8a7176d40441","Type":"ContainerDied","Data":"dddb54d6de96ed78ff5f0831b97f3e21e94a7745b11a6aac6a05832d486ebd76"} Jul 24 13:41:45.569390 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.569367 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm" event={"ID":"2a16ceb0-9322-4f1e-b832-434602ebaf83","Type":"ContainerDied","Data":"f1d6ed54351e7255ec6260d43bf3105435a86c3229e7c30be37720dce269c86c"} Jul 24 13:41:45.569506 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.569410 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1d6ed54351e7255ec6260d43bf3105435a86c3229e7c30be37720dce269c86c" Jul 24 13:41:45.570215 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.569691 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm" Jul 24 13:41:45.704248 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.704219 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf" Jul 24 13:41:45.738325 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.738301 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf" Jul 24 13:41:45.800387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.800354 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b9f13076-7dba-4f74-8cbd-7be65860bb40-util\") pod \"b9f13076-7dba-4f74-8cbd-7be65860bb40\" (UID: \"b9f13076-7dba-4f74-8cbd-7be65860bb40\") " Jul 24 13:41:45.800387 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.800388 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqhwh\" (UniqueName: \"kubernetes.io/projected/b9f13076-7dba-4f74-8cbd-7be65860bb40-kube-api-access-tqhwh\") pod \"b9f13076-7dba-4f74-8cbd-7be65860bb40\" (UID: \"b9f13076-7dba-4f74-8cbd-7be65860bb40\") " Jul 24 13:41:45.800619 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.800435 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3174010-56ea-472b-b6c0-f74cb8bcde98-bundle\") pod \"d3174010-56ea-472b-b6c0-f74cb8bcde98\" (UID: \"d3174010-56ea-472b-b6c0-f74cb8bcde98\") " Jul 24 13:41:45.800619 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.800463 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nbv5\" (UniqueName: \"kubernetes.io/projected/d3174010-56ea-472b-b6c0-f74cb8bcde98-kube-api-access-5nbv5\") pod \"d3174010-56ea-472b-b6c0-f74cb8bcde98\" (UID: \"d3174010-56ea-472b-b6c0-f74cb8bcde98\") " Jul 24 13:41:45.800619 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.800488 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b9f13076-7dba-4f74-8cbd-7be65860bb40-bundle\") pod \"b9f13076-7dba-4f74-8cbd-7be65860bb40\" (UID: \"b9f13076-7dba-4f74-8cbd-7be65860bb40\") " Jul 24 13:41:45.800619 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.800510 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3174010-56ea-472b-b6c0-f74cb8bcde98-util\") pod \"d3174010-56ea-472b-b6c0-f74cb8bcde98\" (UID: \"d3174010-56ea-472b-b6c0-f74cb8bcde98\") " Jul 24 13:41:45.801123 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.801084 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9f13076-7dba-4f74-8cbd-7be65860bb40-bundle" (OuterVolumeSpecName: "bundle") pod "b9f13076-7dba-4f74-8cbd-7be65860bb40" (UID: "b9f13076-7dba-4f74-8cbd-7be65860bb40"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:41:45.801123 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.801119 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3174010-56ea-472b-b6c0-f74cb8bcde98-bundle" (OuterVolumeSpecName: "bundle") pod "d3174010-56ea-472b-b6c0-f74cb8bcde98" (UID: "d3174010-56ea-472b-b6c0-f74cb8bcde98"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:41:45.802779 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.802757 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3174010-56ea-472b-b6c0-f74cb8bcde98-kube-api-access-5nbv5" (OuterVolumeSpecName: "kube-api-access-5nbv5") pod "d3174010-56ea-472b-b6c0-f74cb8bcde98" (UID: "d3174010-56ea-472b-b6c0-f74cb8bcde98"). InnerVolumeSpecName "kube-api-access-5nbv5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:41:45.803133 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.803098 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9f13076-7dba-4f74-8cbd-7be65860bb40-kube-api-access-tqhwh" (OuterVolumeSpecName: "kube-api-access-tqhwh") pod "b9f13076-7dba-4f74-8cbd-7be65860bb40" (UID: "b9f13076-7dba-4f74-8cbd-7be65860bb40"). InnerVolumeSpecName "kube-api-access-tqhwh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:41:45.806612 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.806569 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3174010-56ea-472b-b6c0-f74cb8bcde98-util" (OuterVolumeSpecName: "util") pod "d3174010-56ea-472b-b6c0-f74cb8bcde98" (UID: "d3174010-56ea-472b-b6c0-f74cb8bcde98"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:41:45.807077 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.807055 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9f13076-7dba-4f74-8cbd-7be65860bb40-util" (OuterVolumeSpecName: "util") pod "b9f13076-7dba-4f74-8cbd-7be65860bb40" (UID: "b9f13076-7dba-4f74-8cbd-7be65860bb40"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:41:45.901426 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.901350 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b9f13076-7dba-4f74-8cbd-7be65860bb40-util\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:41:45.901426 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.901375 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqhwh\" (UniqueName: \"kubernetes.io/projected/b9f13076-7dba-4f74-8cbd-7be65860bb40-kube-api-access-tqhwh\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:41:45.901426 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.901386 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3174010-56ea-472b-b6c0-f74cb8bcde98-bundle\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:41:45.901426 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.901395 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nbv5\" (UniqueName: \"kubernetes.io/projected/d3174010-56ea-472b-b6c0-f74cb8bcde98-kube-api-access-5nbv5\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:41:45.901426 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.901404 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b9f13076-7dba-4f74-8cbd-7be65860bb40-bundle\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:41:45.901426 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:45.901412 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3174010-56ea-472b-b6c0-f74cb8bcde98-util\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:41:46.579592 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:46.579545 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf" event={"ID":"d3174010-56ea-472b-b6c0-f74cb8bcde98","Type":"ContainerDied","Data":"47277d40803054338b7f13f603c3cbae50deb119a071da4502c0772a2bdf19b5"} Jul 24 13:41:46.580033 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:46.579870 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="47277d40803054338b7f13f603c3cbae50deb119a071da4502c0772a2bdf19b5" Jul 24 13:41:46.580033 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:46.579561 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf" Jul 24 13:41:46.582325 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:46.582301 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf" Jul 24 13:41:46.582477 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:46.582322 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf" event={"ID":"b9f13076-7dba-4f74-8cbd-7be65860bb40","Type":"ContainerDied","Data":"8bc51dc8362e12fcc704e36a402bab57645df06b0314c47db591efa99f3ea131"} Jul 24 13:41:46.582477 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:46.582379 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8bc51dc8362e12fcc704e36a402bab57645df06b0314c47db591efa99f3ea131" Jul 24 13:41:46.711945 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:46.711916 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf" Jul 24 13:41:46.810666 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:46.810632 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thzsj\" (UniqueName: \"kubernetes.io/projected/10be9861-183d-4216-8429-8a7176d40441-kube-api-access-thzsj\") pod \"10be9861-183d-4216-8429-8a7176d40441\" (UID: \"10be9861-183d-4216-8429-8a7176d40441\") " Jul 24 13:41:46.810817 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:46.810693 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10be9861-183d-4216-8429-8a7176d40441-bundle\") pod \"10be9861-183d-4216-8429-8a7176d40441\" (UID: \"10be9861-183d-4216-8429-8a7176d40441\") " Jul 24 13:41:46.810817 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:46.810769 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10be9861-183d-4216-8429-8a7176d40441-util\") pod \"10be9861-183d-4216-8429-8a7176d40441\" (UID: \"10be9861-183d-4216-8429-8a7176d40441\") " Jul 24 13:41:46.811351 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:46.811320 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10be9861-183d-4216-8429-8a7176d40441-bundle" (OuterVolumeSpecName: "bundle") pod "10be9861-183d-4216-8429-8a7176d40441" (UID: "10be9861-183d-4216-8429-8a7176d40441"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:41:46.812876 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:46.812851 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10be9861-183d-4216-8429-8a7176d40441-kube-api-access-thzsj" (OuterVolumeSpecName: "kube-api-access-thzsj") pod "10be9861-183d-4216-8429-8a7176d40441" (UID: "10be9861-183d-4216-8429-8a7176d40441"). InnerVolumeSpecName "kube-api-access-thzsj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:41:46.816414 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:46.816376 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10be9861-183d-4216-8429-8a7176d40441-util" (OuterVolumeSpecName: "util") pod "10be9861-183d-4216-8429-8a7176d40441" (UID: "10be9861-183d-4216-8429-8a7176d40441"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:41:46.911466 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:46.911425 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thzsj\" (UniqueName: \"kubernetes.io/projected/10be9861-183d-4216-8429-8a7176d40441-kube-api-access-thzsj\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:41:46.911466 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:46.911460 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10be9861-183d-4216-8429-8a7176d40441-bundle\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:41:46.911466 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:46.911471 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10be9861-183d-4216-8429-8a7176d40441-util\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:41:47.589075 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:47.589042 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf" event={"ID":"10be9861-183d-4216-8429-8a7176d40441","Type":"ContainerDied","Data":"fdedc815f20ac8090ecb78e2bf7c3655f0157d7000c6a67d2299a07488e9b761"} Jul 24 13:41:47.589075 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:47.589060 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf" Jul 24 13:41:47.589583 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:47.589088 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fdedc815f20ac8090ecb78e2bf7c3655f0157d7000c6a67d2299a07488e9b761" Jul 24 13:41:53.466803 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:53.466758 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:54.072759 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:54.072722 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:58.404850 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:58.404799 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:59.095321 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:59.095283 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:59.239616 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:59.239573 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:59.385005 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:41:59.384885 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:01.610697 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:01.610657 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:02.563470 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.563443 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8"] Jul 24 13:42:02.563843 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.563826 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10be9861-183d-4216-8429-8a7176d40441" containerName="util" Jul 24 13:42:02.563929 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.563845 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="10be9861-183d-4216-8429-8a7176d40441" containerName="util" Jul 24 13:42:02.563929 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.563858 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3174010-56ea-472b-b6c0-f74cb8bcde98" containerName="pull" Jul 24 13:42:02.563929 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.563866 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3174010-56ea-472b-b6c0-f74cb8bcde98" containerName="pull" Jul 24 13:42:02.563929 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.563883 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a16ceb0-9322-4f1e-b832-434602ebaf83" containerName="pull" Jul 24 13:42:02.563929 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.563892 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a16ceb0-9322-4f1e-b832-434602ebaf83" containerName="pull" Jul 24 13:42:02.563929 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.563901 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10be9861-183d-4216-8429-8a7176d40441" containerName="extract" Jul 24 13:42:02.563929 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.563910 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="10be9861-183d-4216-8429-8a7176d40441" containerName="extract" Jul 24 13:42:02.563929 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.563926 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3174010-56ea-472b-b6c0-f74cb8bcde98" containerName="extract" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.563934 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3174010-56ea-472b-b6c0-f74cb8bcde98" containerName="extract" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.563946 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10be9861-183d-4216-8429-8a7176d40441" containerName="pull" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.563954 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="10be9861-183d-4216-8429-8a7176d40441" containerName="pull" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.563968 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a16ceb0-9322-4f1e-b832-434602ebaf83" containerName="util" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.563976 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a16ceb0-9322-4f1e-b832-434602ebaf83" containerName="util" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.563985 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9f13076-7dba-4f74-8cbd-7be65860bb40" containerName="util" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.564013 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9f13076-7dba-4f74-8cbd-7be65860bb40" containerName="util" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.564030 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3174010-56ea-472b-b6c0-f74cb8bcde98" containerName="util" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.564038 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3174010-56ea-472b-b6c0-f74cb8bcde98" containerName="util" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.564051 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9f13076-7dba-4f74-8cbd-7be65860bb40" containerName="extract" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.564059 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9f13076-7dba-4f74-8cbd-7be65860bb40" containerName="extract" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.564071 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a16ceb0-9322-4f1e-b832-434602ebaf83" containerName="extract" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.564079 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a16ceb0-9322-4f1e-b832-434602ebaf83" containerName="extract" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.564089 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9f13076-7dba-4f74-8cbd-7be65860bb40" containerName="pull" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.564096 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9f13076-7dba-4f74-8cbd-7be65860bb40" containerName="pull" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.564189 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3174010-56ea-472b-b6c0-f74cb8bcde98" containerName="extract" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.564203 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="10be9861-183d-4216-8429-8a7176d40441" containerName="extract" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.564213 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9f13076-7dba-4f74-8cbd-7be65860bb40" containerName="extract" Jul 24 13:42:02.564335 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.564224 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a16ceb0-9322-4f1e-b832-434602ebaf83" containerName="extract" Jul 24 13:42:02.569358 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.569335 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8" Jul 24 13:42:02.573467 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.573439 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 24 13:42:02.573592 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.573439 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 24 13:42:02.573592 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.573443 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-p287l\"" Jul 24 13:42:02.584880 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.584856 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8"] Jul 24 13:42:02.754112 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.754059 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5tz5\" (UniqueName: \"kubernetes.io/projected/fcb21131-6420-4d19-9bd6-84e269be6152-kube-api-access-n5tz5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m4tt8\" (UID: \"fcb21131-6420-4d19-9bd6-84e269be6152\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8" Jul 24 13:42:02.754519 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.754140 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcb21131-6420-4d19-9bd6-84e269be6152-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m4tt8\" (UID: \"fcb21131-6420-4d19-9bd6-84e269be6152\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8" Jul 24 13:42:02.855612 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.855516 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5tz5\" (UniqueName: \"kubernetes.io/projected/fcb21131-6420-4d19-9bd6-84e269be6152-kube-api-access-n5tz5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m4tt8\" (UID: \"fcb21131-6420-4d19-9bd6-84e269be6152\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8" Jul 24 13:42:02.855612 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.855584 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcb21131-6420-4d19-9bd6-84e269be6152-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m4tt8\" (UID: \"fcb21131-6420-4d19-9bd6-84e269be6152\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8" Jul 24 13:42:02.855911 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.855890 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcb21131-6420-4d19-9bd6-84e269be6152-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m4tt8\" (UID: \"fcb21131-6420-4d19-9bd6-84e269be6152\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8" Jul 24 13:42:02.866206 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.866185 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5tz5\" (UniqueName: \"kubernetes.io/projected/fcb21131-6420-4d19-9bd6-84e269be6152-kube-api-access-n5tz5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m4tt8\" (UID: \"fcb21131-6420-4d19-9bd6-84e269be6152\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8" Jul 24 13:42:02.879668 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:02.879643 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8" Jul 24 13:42:03.030213 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:03.030186 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8"] Jul 24 13:42:03.032398 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:42:03.032368 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcb21131_6420_4d19_9bd6_84e269be6152.slice/crio-6016523cc37f48bf73bb94eab5f0a057f74068ad2a797cd03777b5bb386692f0 WatchSource:0}: Error finding container 6016523cc37f48bf73bb94eab5f0a057f74068ad2a797cd03777b5bb386692f0: Status 404 returned error can't find the container with id 6016523cc37f48bf73bb94eab5f0a057f74068ad2a797cd03777b5bb386692f0 Jul 24 13:42:03.087404 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:03.087366 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:03.676268 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:03.676226 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8" event={"ID":"fcb21131-6420-4d19-9bd6-84e269be6152","Type":"ContainerStarted","Data":"6016523cc37f48bf73bb94eab5f0a057f74068ad2a797cd03777b5bb386692f0"} Jul 24 13:42:05.117328 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:05.117288 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:05.209942 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:05.209899 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:07.330634 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:07.330596 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:08.706397 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:08.706308 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8" event={"ID":"fcb21131-6420-4d19-9bd6-84e269be6152","Type":"ContainerStarted","Data":"cac5070e4a48ade1b7c5bdd496cb341dac3f3b4f615175ae91c83a310f45fa3d"} Jul 24 13:42:08.706777 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:08.706511 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8" Jul 24 13:42:08.739158 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:08.739100 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8" podStartSLOduration=1.320780659 podStartE2EDuration="6.73908176s" totalImagesPullingTime="5.418301101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:42:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:42:08.73538648 +0000 UTC m=+470.858768879" watchObservedRunningTime="2026-07-24 13:42:08.73908176 +0000 UTC m=+470.862464217" Jul 24 13:42:09.115078 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:09.115042 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:09.216456 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:09.216421 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:09.482431 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:09.482330 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:09.556640 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:09.556598 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:10.439743 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:10.439710 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:10.540033 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:10.539982 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:10.617787 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:10.617747 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:11.362727 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:11.362692 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:11.462054 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:11.462022 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:14.567929 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:14.567890 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:14.733695 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:14.733664 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:19.713621 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:19.713591 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8" Jul 24 13:42:21.579445 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:21.579404 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:21.696923 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:21.696893 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8"] Jul 24 13:42:21.697139 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:21.697103 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8" podUID="fcb21131-6420-4d19-9bd6-84e269be6152" containerName="manager" containerID="cri-o://cac5070e4a48ade1b7c5bdd496cb341dac3f3b4f615175ae91c83a310f45fa3d" gracePeriod=2 Jul 24 13:42:21.703549 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:21.703522 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8"] Jul 24 13:42:21.726064 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:21.726033 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:21.736607 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:21.736577 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb"] Jul 24 13:42:21.737065 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:21.737043 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcb21131-6420-4d19-9bd6-84e269be6152" containerName="manager" Jul 24 13:42:21.737065 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:21.737064 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcb21131-6420-4d19-9bd6-84e269be6152" containerName="manager" Jul 24 13:42:21.737249 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:21.737141 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcb21131-6420-4d19-9bd6-84e269be6152" containerName="manager" Jul 24 13:42:21.740020 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:21.739968 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb" Jul 24 13:42:21.743141 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:42:21.743114 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m4tt8\" is forbidden: User \"system:node:ip-10-0-137-60.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-60.ec2.internal' and this object" podUID="fcb21131-6420-4d19-9bd6-84e269be6152" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8" Jul 24 13:42:21.761301 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:21.761274 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb"] Jul 24 13:42:21.822158 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:21.822125 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7d48311-a5b5-492a-b99b-2495bb1355e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rq9jb\" (UID: \"e7d48311-a5b5-492a-b99b-2495bb1355e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb" Jul 24 13:42:21.822295 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:21.822190 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t4kw\" (UniqueName: \"kubernetes.io/projected/e7d48311-a5b5-492a-b99b-2495bb1355e8-kube-api-access-2t4kw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rq9jb\" (UID: \"e7d48311-a5b5-492a-b99b-2495bb1355e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb" Jul 24 13:42:21.923601 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:21.923574 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7d48311-a5b5-492a-b99b-2495bb1355e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rq9jb\" (UID: \"e7d48311-a5b5-492a-b99b-2495bb1355e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb" Jul 24 13:42:21.923750 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:21.923641 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t4kw\" (UniqueName: \"kubernetes.io/projected/e7d48311-a5b5-492a-b99b-2495bb1355e8-kube-api-access-2t4kw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rq9jb\" (UID: \"e7d48311-a5b5-492a-b99b-2495bb1355e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb" Jul 24 13:42:21.924007 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:21.923961 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7d48311-a5b5-492a-b99b-2495bb1355e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rq9jb\" (UID: \"e7d48311-a5b5-492a-b99b-2495bb1355e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb" Jul 24 13:42:21.936248 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:21.936223 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t4kw\" (UniqueName: \"kubernetes.io/projected/e7d48311-a5b5-492a-b99b-2495bb1355e8-kube-api-access-2t4kw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rq9jb\" (UID: \"e7d48311-a5b5-492a-b99b-2495bb1355e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb" Jul 24 13:42:21.939840 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:21.939821 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8" Jul 24 13:42:21.942544 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:42:21.942514 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m4tt8\" is forbidden: User \"system:node:ip-10-0-137-60.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-60.ec2.internal' and this object" podUID="fcb21131-6420-4d19-9bd6-84e269be6152" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8" Jul 24 13:42:22.024788 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.024758 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5tz5\" (UniqueName: \"kubernetes.io/projected/fcb21131-6420-4d19-9bd6-84e269be6152-kube-api-access-n5tz5\") pod \"fcb21131-6420-4d19-9bd6-84e269be6152\" (UID: \"fcb21131-6420-4d19-9bd6-84e269be6152\") " Jul 24 13:42:22.024955 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.024829 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcb21131-6420-4d19-9bd6-84e269be6152-extensions-socket-volume\") pod \"fcb21131-6420-4d19-9bd6-84e269be6152\" (UID: \"fcb21131-6420-4d19-9bd6-84e269be6152\") " Jul 24 13:42:22.025259 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.025235 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcb21131-6420-4d19-9bd6-84e269be6152-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fcb21131-6420-4d19-9bd6-84e269be6152" (UID: "fcb21131-6420-4d19-9bd6-84e269be6152"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:42:22.027074 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.027051 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcb21131-6420-4d19-9bd6-84e269be6152-kube-api-access-n5tz5" (OuterVolumeSpecName: "kube-api-access-n5tz5") pod "fcb21131-6420-4d19-9bd6-84e269be6152" (UID: "fcb21131-6420-4d19-9bd6-84e269be6152"). InnerVolumeSpecName "kube-api-access-n5tz5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:42:22.078790 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.078757 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb" Jul 24 13:42:22.126193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.126116 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5tz5\" (UniqueName: \"kubernetes.io/projected/fcb21131-6420-4d19-9bd6-84e269be6152-kube-api-access-n5tz5\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:42:22.126193 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.126147 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcb21131-6420-4d19-9bd6-84e269be6152-extensions-socket-volume\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:42:22.244553 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.244527 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb"] Jul 24 13:42:22.246742 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:42:22.246716 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7d48311_a5b5_492a_b99b_2495bb1355e8.slice/crio-328e448d0e5b819c2d7064acaf8ba3a52a656465d4c3698bfb03698e572f7c58 WatchSource:0}: Error finding container 328e448d0e5b819c2d7064acaf8ba3a52a656465d4c3698bfb03698e572f7c58: Status 404 returned error can't find the container with id 328e448d0e5b819c2d7064acaf8ba3a52a656465d4c3698bfb03698e572f7c58 Jul 24 13:42:22.419698 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.419620 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcb21131-6420-4d19-9bd6-84e269be6152" path="/var/lib/kubelet/pods/fcb21131-6420-4d19-9bd6-84e269be6152/volumes" Jul 24 13:42:22.689858 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.689726 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b"] Jul 24 13:42:22.693879 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.693861 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b" Jul 24 13:42:22.719567 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.719541 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b"] Jul 24 13:42:22.731620 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.731594 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/310b06ce-c5b4-4fb0-bb20-c663da8ebbf4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7664b\" (UID: \"310b06ce-c5b4-4fb0-bb20-c663da8ebbf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b" Jul 24 13:42:22.731759 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.731672 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dd2v\" (UniqueName: \"kubernetes.io/projected/310b06ce-c5b4-4fb0-bb20-c663da8ebbf4-kube-api-access-4dd2v\") pod \"kuadrant-operator-controller-manager-86878c769f-7664b\" (UID: \"310b06ce-c5b4-4fb0-bb20-c663da8ebbf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b" Jul 24 13:42:22.780412 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.780379 2579 generic.go:356] "Generic (PLEG): container finished" podID="fcb21131-6420-4d19-9bd6-84e269be6152" containerID="cac5070e4a48ade1b7c5bdd496cb341dac3f3b4f615175ae91c83a310f45fa3d" exitCode=0 Jul 24 13:42:22.780582 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.780431 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4tt8" Jul 24 13:42:22.780582 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.780475 2579 scope.go:117] "RemoveContainer" containerID="cac5070e4a48ade1b7c5bdd496cb341dac3f3b4f615175ae91c83a310f45fa3d" Jul 24 13:42:22.782554 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.782527 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb" event={"ID":"e7d48311-a5b5-492a-b99b-2495bb1355e8","Type":"ContainerStarted","Data":"d30ab1eeb257299b7c723f3d6061e5537376b59ce45cad24357263a99ad3b51c"} Jul 24 13:42:22.782679 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.782585 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb" event={"ID":"e7d48311-a5b5-492a-b99b-2495bb1355e8","Type":"ContainerStarted","Data":"328e448d0e5b819c2d7064acaf8ba3a52a656465d4c3698bfb03698e572f7c58"} Jul 24 13:42:22.782679 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.782635 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb" Jul 24 13:42:22.789207 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.789192 2579 scope.go:117] "RemoveContainer" containerID="cac5070e4a48ade1b7c5bdd496cb341dac3f3b4f615175ae91c83a310f45fa3d" Jul 24 13:42:22.789476 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:42:22.789453 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cac5070e4a48ade1b7c5bdd496cb341dac3f3b4f615175ae91c83a310f45fa3d\": container with ID starting with cac5070e4a48ade1b7c5bdd496cb341dac3f3b4f615175ae91c83a310f45fa3d not found: ID does not exist" containerID="cac5070e4a48ade1b7c5bdd496cb341dac3f3b4f615175ae91c83a310f45fa3d" Jul 24 13:42:22.789588 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.789481 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cac5070e4a48ade1b7c5bdd496cb341dac3f3b4f615175ae91c83a310f45fa3d"} err="failed to get container status \"cac5070e4a48ade1b7c5bdd496cb341dac3f3b4f615175ae91c83a310f45fa3d\": rpc error: code = NotFound desc = could not find container \"cac5070e4a48ade1b7c5bdd496cb341dac3f3b4f615175ae91c83a310f45fa3d\": container with ID starting with cac5070e4a48ade1b7c5bdd496cb341dac3f3b4f615175ae91c83a310f45fa3d not found: ID does not exist" Jul 24 13:42:22.832540 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.832504 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dd2v\" (UniqueName: \"kubernetes.io/projected/310b06ce-c5b4-4fb0-bb20-c663da8ebbf4-kube-api-access-4dd2v\") pod \"kuadrant-operator-controller-manager-86878c769f-7664b\" (UID: \"310b06ce-c5b4-4fb0-bb20-c663da8ebbf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b" Jul 24 13:42:22.832739 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.832616 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/310b06ce-c5b4-4fb0-bb20-c663da8ebbf4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7664b\" (UID: \"310b06ce-c5b4-4fb0-bb20-c663da8ebbf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b" Jul 24 13:42:22.833051 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.833030 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/310b06ce-c5b4-4fb0-bb20-c663da8ebbf4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7664b\" (UID: \"310b06ce-c5b4-4fb0-bb20-c663da8ebbf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b" Jul 24 13:42:22.866339 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:22.866310 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dd2v\" (UniqueName: \"kubernetes.io/projected/310b06ce-c5b4-4fb0-bb20-c663da8ebbf4-kube-api-access-4dd2v\") pod \"kuadrant-operator-controller-manager-86878c769f-7664b\" (UID: \"310b06ce-c5b4-4fb0-bb20-c663da8ebbf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b" Jul 24 13:42:23.003502 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:23.003415 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b" Jul 24 13:42:23.151385 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:23.151319 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb" podStartSLOduration=2.151296515 podStartE2EDuration="2.151296515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:42:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:42:22.83131229 +0000 UTC m=+484.954694678" watchObservedRunningTime="2026-07-24 13:42:23.151296515 +0000 UTC m=+485.274678913" Jul 24 13:42:23.152116 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:23.152088 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b"] Jul 24 13:42:23.157714 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:42:23.157682 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod310b06ce_c5b4_4fb0_bb20_c663da8ebbf4.slice/crio-71e3b908e793fafd703df5483f9b1cfea622eac9d62d60b171b4cfc3791ba1ac WatchSource:0}: Error finding container 71e3b908e793fafd703df5483f9b1cfea622eac9d62d60b171b4cfc3791ba1ac: Status 404 returned error can't find the container with id 71e3b908e793fafd703df5483f9b1cfea622eac9d62d60b171b4cfc3791ba1ac Jul 24 13:42:23.789828 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:23.789781 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b" event={"ID":"310b06ce-c5b4-4fb0-bb20-c663da8ebbf4","Type":"ContainerStarted","Data":"425c094646840d4a5375c5a3c52ae20bc7e86de0860aa87b4417986a942c2dc0"} Jul 24 13:42:23.790269 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:23.789853 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b" event={"ID":"310b06ce-c5b4-4fb0-bb20-c663da8ebbf4","Type":"ContainerStarted","Data":"71e3b908e793fafd703df5483f9b1cfea622eac9d62d60b171b4cfc3791ba1ac"} Jul 24 13:42:23.790269 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:23.789885 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b" Jul 24 13:42:23.824331 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:23.824280 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b" podStartSLOduration=1.824264399 podStartE2EDuration="1.824264399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:42:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:42:23.823253489 +0000 UTC m=+485.946635887" watchObservedRunningTime="2026-07-24 13:42:23.824264399 +0000 UTC m=+485.947646798" Jul 24 13:42:23.986647 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:23.986590 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:24.082916 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:24.082855 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:29.699012 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:29.698963 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:29.772660 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:29.772626 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:30.293087 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:30.293048 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:30.413653 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:30.413616 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:33.693695 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:33.693659 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:33.756425 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:33.756388 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:33.793380 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:33.793350 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb" Jul 24 13:42:33.912873 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:33.912842 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:33.985811 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:33.985723 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:34.798836 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:34.798808 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b" Jul 24 13:42:34.857830 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:34.857800 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb"] Jul 24 13:42:34.858122 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:34.858080 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb" podUID="e7d48311-a5b5-492a-b99b-2495bb1355e8" containerName="manager" containerID="cri-o://d30ab1eeb257299b7c723f3d6061e5537376b59ce45cad24357263a99ad3b51c" gracePeriod=10 Jul 24 13:42:35.145013 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:35.144962 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb" Jul 24 13:42:35.242791 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:35.242758 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7d48311-a5b5-492a-b99b-2495bb1355e8-extensions-socket-volume\") pod \"e7d48311-a5b5-492a-b99b-2495bb1355e8\" (UID: \"e7d48311-a5b5-492a-b99b-2495bb1355e8\") " Jul 24 13:42:35.242966 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:35.242916 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2t4kw\" (UniqueName: \"kubernetes.io/projected/e7d48311-a5b5-492a-b99b-2495bb1355e8-kube-api-access-2t4kw\") pod \"e7d48311-a5b5-492a-b99b-2495bb1355e8\" (UID: \"e7d48311-a5b5-492a-b99b-2495bb1355e8\") " Jul 24 13:42:35.243263 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:35.243243 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7d48311-a5b5-492a-b99b-2495bb1355e8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e7d48311-a5b5-492a-b99b-2495bb1355e8" (UID: "e7d48311-a5b5-492a-b99b-2495bb1355e8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:42:35.245132 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:35.245108 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7d48311-a5b5-492a-b99b-2495bb1355e8-kube-api-access-2t4kw" (OuterVolumeSpecName: "kube-api-access-2t4kw") pod "e7d48311-a5b5-492a-b99b-2495bb1355e8" (UID: "e7d48311-a5b5-492a-b99b-2495bb1355e8"). InnerVolumeSpecName "kube-api-access-2t4kw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:42:35.344415 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:35.344326 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2t4kw\" (UniqueName: \"kubernetes.io/projected/e7d48311-a5b5-492a-b99b-2495bb1355e8-kube-api-access-2t4kw\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:42:35.344415 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:35.344354 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7d48311-a5b5-492a-b99b-2495bb1355e8-extensions-socket-volume\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:42:35.857458 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:35.857420 2579 generic.go:356] "Generic (PLEG): container finished" podID="e7d48311-a5b5-492a-b99b-2495bb1355e8" containerID="d30ab1eeb257299b7c723f3d6061e5537376b59ce45cad24357263a99ad3b51c" exitCode=0 Jul 24 13:42:35.857937 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:35.857474 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb" Jul 24 13:42:35.857937 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:35.857518 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb" event={"ID":"e7d48311-a5b5-492a-b99b-2495bb1355e8","Type":"ContainerDied","Data":"d30ab1eeb257299b7c723f3d6061e5537376b59ce45cad24357263a99ad3b51c"} Jul 24 13:42:35.857937 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:35.857581 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb" event={"ID":"e7d48311-a5b5-492a-b99b-2495bb1355e8","Type":"ContainerDied","Data":"328e448d0e5b819c2d7064acaf8ba3a52a656465d4c3698bfb03698e572f7c58"} Jul 24 13:42:35.857937 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:35.857618 2579 scope.go:117] "RemoveContainer" containerID="d30ab1eeb257299b7c723f3d6061e5537376b59ce45cad24357263a99ad3b51c" Jul 24 13:42:35.867280 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:35.867264 2579 scope.go:117] "RemoveContainer" containerID="d30ab1eeb257299b7c723f3d6061e5537376b59ce45cad24357263a99ad3b51c" Jul 24 13:42:35.867549 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:42:35.867531 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d30ab1eeb257299b7c723f3d6061e5537376b59ce45cad24357263a99ad3b51c\": container with ID starting with d30ab1eeb257299b7c723f3d6061e5537376b59ce45cad24357263a99ad3b51c not found: ID does not exist" containerID="d30ab1eeb257299b7c723f3d6061e5537376b59ce45cad24357263a99ad3b51c" Jul 24 13:42:35.867588 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:35.867559 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d30ab1eeb257299b7c723f3d6061e5537376b59ce45cad24357263a99ad3b51c"} err="failed to get container status \"d30ab1eeb257299b7c723f3d6061e5537376b59ce45cad24357263a99ad3b51c\": rpc error: code = NotFound desc = could not find container \"d30ab1eeb257299b7c723f3d6061e5537376b59ce45cad24357263a99ad3b51c\": container with ID starting with d30ab1eeb257299b7c723f3d6061e5537376b59ce45cad24357263a99ad3b51c not found: ID does not exist" Jul 24 13:42:35.884159 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:35.884131 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb"] Jul 24 13:42:35.888880 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:35.888861 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rq9jb"] Jul 24 13:42:36.419316 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:36.419281 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7d48311-a5b5-492a-b99b-2495bb1355e8" path="/var/lib/kubelet/pods/e7d48311-a5b5-492a-b99b-2495bb1355e8/volumes" Jul 24 13:42:41.300294 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:41.300258 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:41.403625 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:41.403590 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:43.581535 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:43.581489 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:43.655675 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:43.655638 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:51.192346 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.192294 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m"] Jul 24 13:42:51.192854 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.192834 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7d48311-a5b5-492a-b99b-2495bb1355e8" containerName="manager" Jul 24 13:42:51.192928 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.192856 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7d48311-a5b5-492a-b99b-2495bb1355e8" containerName="manager" Jul 24 13:42:51.193012 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.192952 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7d48311-a5b5-492a-b99b-2495bb1355e8" containerName="manager" Jul 24 13:42:51.196131 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.196106 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.199362 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.199332 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-ghgl7\"" Jul 24 13:42:51.208968 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.208943 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m"] Jul 24 13:42:51.222460 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.222429 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:51.291594 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.291553 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0557c33f-f425-4707-90fc-085902eac820-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.291594 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.291598 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk7t2\" (UniqueName: \"kubernetes.io/projected/0557c33f-f425-4707-90fc-085902eac820-kube-api-access-pk7t2\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.291823 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.291681 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0557c33f-f425-4707-90fc-085902eac820-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.291823 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.291763 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0557c33f-f425-4707-90fc-085902eac820-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.291823 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.291789 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0557c33f-f425-4707-90fc-085902eac820-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.291823 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.291809 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0557c33f-f425-4707-90fc-085902eac820-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.291979 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.291827 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0557c33f-f425-4707-90fc-085902eac820-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.291979 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.291873 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0557c33f-f425-4707-90fc-085902eac820-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.291979 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.291907 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0557c33f-f425-4707-90fc-085902eac820-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.322680 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.322631 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:51.392764 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.392726 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0557c33f-f425-4707-90fc-085902eac820-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.392764 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.392766 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0557c33f-f425-4707-90fc-085902eac820-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.393069 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.392794 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0557c33f-f425-4707-90fc-085902eac820-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.393069 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.392821 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0557c33f-f425-4707-90fc-085902eac820-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.393069 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.392848 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0557c33f-f425-4707-90fc-085902eac820-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.393069 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.392871 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0557c33f-f425-4707-90fc-085902eac820-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.393069 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.392964 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0557c33f-f425-4707-90fc-085902eac820-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.393069 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.393027 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk7t2\" (UniqueName: \"kubernetes.io/projected/0557c33f-f425-4707-90fc-085902eac820-kube-api-access-pk7t2\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.393395 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.393078 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0557c33f-f425-4707-90fc-085902eac820-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.393395 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.393227 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0557c33f-f425-4707-90fc-085902eac820-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.393525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.393397 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0557c33f-f425-4707-90fc-085902eac820-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.393525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.393400 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0557c33f-f425-4707-90fc-085902eac820-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.393525 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.393503 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0557c33f-f425-4707-90fc-085902eac820-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.393789 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.393759 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0557c33f-f425-4707-90fc-085902eac820-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.395545 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.395517 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0557c33f-f425-4707-90fc-085902eac820-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.395650 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.395609 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0557c33f-f425-4707-90fc-085902eac820-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.402136 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.402114 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0557c33f-f425-4707-90fc-085902eac820-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.402261 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.402225 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk7t2\" (UniqueName: \"kubernetes.io/projected/0557c33f-f425-4707-90fc-085902eac820-kube-api-access-pk7t2\") pod \"maas-default-gateway-openshift-default-78bb78ff78-l5w2m\" (UID: \"0557c33f-f425-4707-90fc-085902eac820\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.507554 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.507469 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:51.652221 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.652190 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m"] Jul 24 13:42:51.653806 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:42:51.653781 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0557c33f_f425_4707_90fc_085902eac820.slice/crio-7054207566edb09a27a72fb1e105bcd1d93891080c7d258fbc68ddc9fd63bb7a WatchSource:0}: Error finding container 7054207566edb09a27a72fb1e105bcd1d93891080c7d258fbc68ddc9fd63bb7a: Status 404 returned error can't find the container with id 7054207566edb09a27a72fb1e105bcd1d93891080c7d258fbc68ddc9fd63bb7a Jul 24 13:42:51.655927 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.655886 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:42:51.656028 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.655979 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:42:51.656076 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.656046 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:42:51.943664 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.943604 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" event={"ID":"0557c33f-f425-4707-90fc-085902eac820","Type":"ContainerStarted","Data":"f0b72b39e9ce9da5a9ceb5a4ae2014fe7594f88d370f20975fdbdef6adfcab53"} Jul 24 13:42:51.943835 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.943676 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" event={"ID":"0557c33f-f425-4707-90fc-085902eac820","Type":"ContainerStarted","Data":"7054207566edb09a27a72fb1e105bcd1d93891080c7d258fbc68ddc9fd63bb7a"} Jul 24 13:42:51.969644 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:51.969584 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" podStartSLOduration=0.969565412 podStartE2EDuration="969.565412ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:42:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:42:51.965676052 +0000 UTC m=+514.089058472" watchObservedRunningTime="2026-07-24 13:42:51.969565412 +0000 UTC m=+514.092947811" Jul 24 13:42:52.508257 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:52.508223 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:52.512874 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:52.512849 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:52.949479 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:52.949447 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:52.950556 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:52.950537 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:42:53.090096 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.090060 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:53.188471 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.188438 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:53.251044 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.250912 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:53.348850 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.348818 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:42:53.352056 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.352034 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-pqc45" Jul 24 13:42:53.354949 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.354925 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wbnz6\"" Jul 24 13:42:53.355113 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.354924 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 24 13:42:53.361274 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.361249 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:42:53.389590 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.389562 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:42:53.514247 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.514151 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z8vh\" (UniqueName: \"kubernetes.io/projected/a0e8e60d-dce0-49c3-b42a-a091b9a0eb2a-kube-api-access-2z8vh\") pod \"limitador-limitador-54d6d988d5-pqc45\" (UID: \"a0e8e60d-dce0-49c3-b42a-a091b9a0eb2a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pqc45" Jul 24 13:42:53.514247 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.514195 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a0e8e60d-dce0-49c3-b42a-a091b9a0eb2a-config-file\") pod \"limitador-limitador-54d6d988d5-pqc45\" (UID: \"a0e8e60d-dce0-49c3-b42a-a091b9a0eb2a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pqc45" Jul 24 13:42:53.615150 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.615116 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z8vh\" (UniqueName: \"kubernetes.io/projected/a0e8e60d-dce0-49c3-b42a-a091b9a0eb2a-kube-api-access-2z8vh\") pod \"limitador-limitador-54d6d988d5-pqc45\" (UID: \"a0e8e60d-dce0-49c3-b42a-a091b9a0eb2a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pqc45" Jul 24 13:42:53.615322 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.615157 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a0e8e60d-dce0-49c3-b42a-a091b9a0eb2a-config-file\") pod \"limitador-limitador-54d6d988d5-pqc45\" (UID: \"a0e8e60d-dce0-49c3-b42a-a091b9a0eb2a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pqc45" Jul 24 13:42:53.615715 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.615693 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a0e8e60d-dce0-49c3-b42a-a091b9a0eb2a-config-file\") pod \"limitador-limitador-54d6d988d5-pqc45\" (UID: \"a0e8e60d-dce0-49c3-b42a-a091b9a0eb2a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pqc45" Jul 24 13:42:53.624907 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.624881 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z8vh\" (UniqueName: \"kubernetes.io/projected/a0e8e60d-dce0-49c3-b42a-a091b9a0eb2a-kube-api-access-2z8vh\") pod \"limitador-limitador-54d6d988d5-pqc45\" (UID: \"a0e8e60d-dce0-49c3-b42a-a091b9a0eb2a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pqc45" Jul 24 13:42:53.664654 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.664621 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-pqc45" Jul 24 13:42:53.812962 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.812938 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:42:53.814931 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:42:53.814911 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e8e60d_dce0_49c3_b42a_a091b9a0eb2a.slice/crio-c9135c4d4f3948d541d73cd34047f0b3a05fc0ef78b01c8e7e143131c4d99ec3 WatchSource:0}: Error finding container c9135c4d4f3948d541d73cd34047f0b3a05fc0ef78b01c8e7e143131c4d99ec3: Status 404 returned error can't find the container with id c9135c4d4f3948d541d73cd34047f0b3a05fc0ef78b01c8e7e143131c4d99ec3 Jul 24 13:42:53.931747 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.931710 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:53.955351 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.955320 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-pqc45" event={"ID":"a0e8e60d-dce0-49c3-b42a-a091b9a0eb2a","Type":"ContainerStarted","Data":"c9135c4d4f3948d541d73cd34047f0b3a05fc0ef78b01c8e7e143131c4d99ec3"} Jul 24 13:42:53.998118 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:53.998084 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:56.974168 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:56.974119 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-pqc45" event={"ID":"a0e8e60d-dce0-49c3-b42a-a091b9a0eb2a","Type":"ContainerStarted","Data":"1bbb84129b1b28134c20cffecf8f8e752b6d2cb468176772798321ece930b51e"} Jul 24 13:42:56.974539 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:56.974255 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-pqc45" Jul 24 13:42:56.993497 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:42:56.993445 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-pqc45" podStartSLOduration=1.4324417999999999 podStartE2EDuration="3.993429928s" totalImagesPullingTime="2.560988128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:42:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:42:56.992245538 +0000 UTC m=+519.115627939" watchObservedRunningTime="2026-07-24 13:42:56.993429928 +0000 UTC m=+519.116812332" Jul 24 13:43:04.246512 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:43:04.246471 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:07.980826 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:43:07.980798 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-pqc45" Jul 24 13:43:09.243508 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:43:09.243408 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:09.348355 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:43:09.348310 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:09.707116 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:43:09.707080 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:21.562768 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:43:21.562731 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:22.092406 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:43:22.092367 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:28.371092 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:43:28.371050 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:28.461481 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:43:28.461445 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:28.543217 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:43:28.543184 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:28.609189 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:43:28.609155 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:32.021844 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:43:32.021795 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:32.121533 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:43:32.121490 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:36.429792 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:43:36.429756 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:03.764472 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:03.764431 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:03.863757 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:03.863718 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:04.309836 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:04.309799 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:04.415850 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:04.415804 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:05.053466 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:05.053423 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:05.157324 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:05.157292 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:05.470121 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:05.470088 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:44:13.400054 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:13.399982 2579 generic.go:356] "Generic (PLEG): container finished" podID="0557c33f-f425-4707-90fc-085902eac820" containerID="f0b72b39e9ce9da5a9ceb5a4ae2014fe7594f88d370f20975fdbdef6adfcab53" exitCode=0 Jul 24 13:44:13.400581 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:13.400063 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" event={"ID":"0557c33f-f425-4707-90fc-085902eac820","Type":"ContainerDied","Data":"f0b72b39e9ce9da5a9ceb5a4ae2014fe7594f88d370f20975fdbdef6adfcab53"} Jul 24 13:44:13.400650 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:13.400583 2579 scope.go:117] "RemoveContainer" containerID="f0b72b39e9ce9da5a9ceb5a4ae2014fe7594f88d370f20975fdbdef6adfcab53" Jul 24 13:44:13.401625 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:13.401599 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:44:13.401693 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:13.401668 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:44:13.401747 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:13.401699 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:44:14.410048 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:14.410013 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" event={"ID":"0557c33f-f425-4707-90fc-085902eac820","Type":"ContainerStarted","Data":"7a0a6e39f6b70aeeb02ca8c04342adfc0373f66af52d8636ab72a308d345d208"} Jul 24 13:44:14.507616 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:14.507576 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:44:14.507912 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:14.507890 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-l5w2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 24 13:44:14.507974 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:14.507955 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" podUID="0557c33f-f425-4707-90fc-085902eac820" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 24 13:44:15.508355 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:15.508319 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-l5w2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 24 13:44:15.508846 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:15.508380 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" podUID="0557c33f-f425-4707-90fc-085902eac820" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 24 13:44:15.583399 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:15.583359 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:15.675185 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:15.675154 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:16.508285 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:16.508255 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-l5w2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 24 13:44:16.508468 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:16.508308 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" podUID="0557c33f-f425-4707-90fc-085902eac820" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 24 13:44:17.508136 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:17.508099 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-l5w2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 24 13:44:17.508315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:17.508152 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" podUID="0557c33f-f425-4707-90fc-085902eac820" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 24 13:44:18.512249 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:18.512214 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:44:18.512716 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:18.512658 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:44:18.513355 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:18.513335 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-l5w2m" Jul 24 13:44:24.086239 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:24.086205 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:24.812745 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:24.812712 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:44:32.791466 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:32.791423 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:39.231412 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:39.231326 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:39.898427 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:39.898392 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:44:43.875418 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:43.875377 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:43.982690 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:43.982640 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:44.377315 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:44.377280 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:45.307341 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:45.307307 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:44:54.010016 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:54.009963 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:55.913807 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:44:55.913769 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:45:00.140461 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:00.140427 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748345-klnkl"] Jul 24 13:45:00.144166 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:00.144141 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748345-klnkl" Jul 24 13:45:00.147107 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:00.147084 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4stw\"" Jul 24 13:45:00.153258 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:00.153221 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748345-klnkl"] Jul 24 13:45:00.157177 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:00.157154 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748345-klnkl" Jul 24 13:45:00.299280 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:00.299251 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748345-klnkl"] Jul 24 13:45:00.301227 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:45:00.301204 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c7372e4_8852_45d3_9879_ceb95f2f0387.slice/crio-d658ce5db12324a2d24e35cc44d0ba014a01e27f6d745253d14dfcd06e0b43a8 WatchSource:0}: Error finding container d658ce5db12324a2d24e35cc44d0ba014a01e27f6d745253d14dfcd06e0b43a8: Status 404 returned error can't find the container with id d658ce5db12324a2d24e35cc44d0ba014a01e27f6d745253d14dfcd06e0b43a8 Jul 24 13:45:00.303141 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:00.303126 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:45:00.657924 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:00.657893 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748345-klnkl" event={"ID":"4c7372e4-8852-45d3-9879-ceb95f2f0387","Type":"ContainerStarted","Data":"d658ce5db12324a2d24e35cc44d0ba014a01e27f6d745253d14dfcd06e0b43a8"} Jul 24 13:45:03.676674 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:03.676577 2579 generic.go:356] "Generic (PLEG): container finished" podID="4c7372e4-8852-45d3-9879-ceb95f2f0387" containerID="0a4cf8513508d84a505b084c8e942e5826354b5204bfb53d90a877a95db734af" exitCode=0 Jul 24 13:45:03.677077 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:03.676672 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748345-klnkl" event={"ID":"4c7372e4-8852-45d3-9879-ceb95f2f0387","Type":"ContainerDied","Data":"0a4cf8513508d84a505b084c8e942e5826354b5204bfb53d90a877a95db734af"} Jul 24 13:45:04.823090 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:04.823057 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748345-klnkl" Jul 24 13:45:05.688485 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:05.688456 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748345-klnkl" Jul 24 13:45:05.688669 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:05.688482 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748345-klnkl" event={"ID":"4c7372e4-8852-45d3-9879-ceb95f2f0387","Type":"ContainerDied","Data":"d658ce5db12324a2d24e35cc44d0ba014a01e27f6d745253d14dfcd06e0b43a8"} Jul 24 13:45:05.688669 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:05.688535 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d658ce5db12324a2d24e35cc44d0ba014a01e27f6d745253d14dfcd06e0b43a8" Jul 24 13:45:09.904288 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:09.904250 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2"] Jul 24 13:45:09.904889 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:09.904817 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c7372e4-8852-45d3-9879-ceb95f2f0387" containerName="cleanup" Jul 24 13:45:09.904889 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:09.904835 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c7372e4-8852-45d3-9879-ceb95f2f0387" containerName="cleanup" Jul 24 13:45:09.904971 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:09.904909 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c7372e4-8852-45d3-9879-ceb95f2f0387" containerName="cleanup" Jul 24 13:45:09.907014 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:09.906968 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:09.911205 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:09.911180 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 24 13:45:09.911403 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:09.911179 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2w2kd\"" Jul 24 13:45:09.911403 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:09.911222 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 24 13:45:09.911403 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:09.911253 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 24 13:45:09.918025 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:09.917985 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2"] Jul 24 13:45:09.967526 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:09.967484 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b6538bb-2afe-462a-a845-8e2348f5155f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-99fz2\" (UID: \"1b6538bb-2afe-462a-a845-8e2348f5155f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:09.967526 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:09.967528 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b6538bb-2afe-462a-a845-8e2348f5155f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-99fz2\" (UID: \"1b6538bb-2afe-462a-a845-8e2348f5155f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:09.967782 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:09.967560 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b6538bb-2afe-462a-a845-8e2348f5155f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-99fz2\" (UID: \"1b6538bb-2afe-462a-a845-8e2348f5155f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:09.967782 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:09.967604 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b6538bb-2afe-462a-a845-8e2348f5155f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-99fz2\" (UID: \"1b6538bb-2afe-462a-a845-8e2348f5155f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:09.967782 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:09.967627 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b6538bb-2afe-462a-a845-8e2348f5155f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-99fz2\" (UID: \"1b6538bb-2afe-462a-a845-8e2348f5155f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:09.967782 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:09.967707 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrgwr\" (UniqueName: \"kubernetes.io/projected/1b6538bb-2afe-462a-a845-8e2348f5155f-kube-api-access-jrgwr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-99fz2\" (UID: \"1b6538bb-2afe-462a-a845-8e2348f5155f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:10.009126 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:10.009049 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:10.068177 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:10.068138 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b6538bb-2afe-462a-a845-8e2348f5155f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-99fz2\" (UID: \"1b6538bb-2afe-462a-a845-8e2348f5155f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:10.068347 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:10.068189 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b6538bb-2afe-462a-a845-8e2348f5155f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-99fz2\" (UID: \"1b6538bb-2afe-462a-a845-8e2348f5155f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:10.068347 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:10.068212 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b6538bb-2afe-462a-a845-8e2348f5155f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-99fz2\" (UID: \"1b6538bb-2afe-462a-a845-8e2348f5155f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:10.068347 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:10.068270 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrgwr\" (UniqueName: \"kubernetes.io/projected/1b6538bb-2afe-462a-a845-8e2348f5155f-kube-api-access-jrgwr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-99fz2\" (UID: \"1b6538bb-2afe-462a-a845-8e2348f5155f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:10.068504 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:10.068355 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b6538bb-2afe-462a-a845-8e2348f5155f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-99fz2\" (UID: \"1b6538bb-2afe-462a-a845-8e2348f5155f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:10.068504 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:10.068377 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b6538bb-2afe-462a-a845-8e2348f5155f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-99fz2\" (UID: \"1b6538bb-2afe-462a-a845-8e2348f5155f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:10.068601 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:10.068540 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b6538bb-2afe-462a-a845-8e2348f5155f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-99fz2\" (UID: \"1b6538bb-2afe-462a-a845-8e2348f5155f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:10.068652 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:10.068598 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b6538bb-2afe-462a-a845-8e2348f5155f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-99fz2\" (UID: \"1b6538bb-2afe-462a-a845-8e2348f5155f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:10.068652 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:10.068637 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b6538bb-2afe-462a-a845-8e2348f5155f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-99fz2\" (UID: \"1b6538bb-2afe-462a-a845-8e2348f5155f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:10.070790 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:10.070758 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b6538bb-2afe-462a-a845-8e2348f5155f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-99fz2\" (UID: \"1b6538bb-2afe-462a-a845-8e2348f5155f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:10.071042 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:10.071025 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b6538bb-2afe-462a-a845-8e2348f5155f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-99fz2\" (UID: \"1b6538bb-2afe-462a-a845-8e2348f5155f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:10.077247 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:10.077220 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrgwr\" (UniqueName: \"kubernetes.io/projected/1b6538bb-2afe-462a-a845-8e2348f5155f-kube-api-access-jrgwr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-99fz2\" (UID: \"1b6538bb-2afe-462a-a845-8e2348f5155f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:10.218266 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:10.218174 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:10.357381 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:10.357350 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2"] Jul 24 13:45:10.359043 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:45:10.359015 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b6538bb_2afe_462a_a845_8e2348f5155f.slice/crio-b47c1de60b7f28ce4fb9ca97c1de84cc3d5a422de4d3ac7f707fd7b64c4b0656 WatchSource:0}: Error finding container b47c1de60b7f28ce4fb9ca97c1de84cc3d5a422de4d3ac7f707fd7b64c4b0656: Status 404 returned error can't find the container with id b47c1de60b7f28ce4fb9ca97c1de84cc3d5a422de4d3ac7f707fd7b64c4b0656 Jul 24 13:45:10.718002 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:10.717959 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" event={"ID":"1b6538bb-2afe-462a-a845-8e2348f5155f","Type":"ContainerStarted","Data":"b47c1de60b7f28ce4fb9ca97c1de84cc3d5a422de4d3ac7f707fd7b64c4b0656"} Jul 24 13:45:11.724735 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:11.724698 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" event={"ID":"1b6538bb-2afe-462a-a845-8e2348f5155f","Type":"ContainerStarted","Data":"3f71005a237314849c0b153d789c3cfc20da2d699bd66efa88d9dff24a4bff79"} Jul 24 13:45:11.747700 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:11.747589 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" podStartSLOduration=1.612693532 podStartE2EDuration="2.747571196s" totalImagesPullingTime="1.134877664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:45:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:45:11.744530864 +0000 UTC m=+653.867913288" watchObservedRunningTime="2026-07-24 13:45:11.747571196 +0000 UTC m=+653.870953594" Jul 24 13:45:12.811586 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:12.811549 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:45:20.218509 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:20.218461 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:20.219091 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:20.218541 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:20.231201 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:20.231177 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:20.790884 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:20.790854 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-99fz2" Jul 24 13:45:22.320812 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:22.320768 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:24.380808 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:24.380771 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:24.814850 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:45:24.814812 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:46:31.363444 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:46:31.363363 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:31.453749 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:46:31.453681 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:32.482617 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:46:32.482566 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:32.575065 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:46:32.574974 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:34.610620 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:46:34.610583 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:01.582940 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:47:01.582904 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:47:02.253322 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:47:02.253248 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:02.352175 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:47:02.352136 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:10.906729 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:47:10.906696 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:47:19.368543 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:47:19.368499 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:19.473150 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:47:19.473118 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:21.226749 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:47:21.226716 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:48:37.330157 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:48:37.330110 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:38.620397 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:48:38.620366 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:04.213458 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:49:04.213419 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:49:12.316963 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:49:12.316929 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:49:14.309777 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:49:14.309733 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:14.460411 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:49:14.460378 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:14.639284 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:49:14.639174 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:20.793043 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:49:20.792982 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:21.150439 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:49:21.150371 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:21.297685 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:49:21.297640 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:27.229354 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:49:27.229315 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:27.330372 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:49:27.330342 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:45.204613 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:49:45.204567 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:49:54.950712 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:49:54.950677 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:50:04.561217 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:50:04.561173 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:50:10.612774 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:50:10.612740 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:50:21.733753 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:50:21.733714 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:50:33.407875 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:50:33.407832 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:50:43.921904 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:50:43.921827 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:50:53.522393 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:50:53.522348 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:51:19.523668 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:51:19.523630 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:51:30.012871 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:51:30.012830 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:51:54.014583 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:51:54.014548 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:52:04.913983 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:52:04.913938 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:52:19.446030 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:52:19.445918 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:52:30.814264 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:52:30.814229 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:52:41.012855 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:52:41.012816 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:52:50.726283 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:52:50.726246 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:53:00.110202 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:00.110164 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:53:14.974779 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:14.974725 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:25.372925 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:25.372883 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b"] Jul 24 13:53:25.373469 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:25.373174 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b" podUID="310b06ce-c5b4-4fb0-bb20-c663da8ebbf4" containerName="manager" containerID="cri-o://425c094646840d4a5375c5a3c52ae20bc7e86de0860aa87b4417986a942c2dc0" gracePeriod=10 Jul 24 13:53:26.292192 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:26.292155 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:28.236264 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:28.236236 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b" Jul 24 13:53:28.264109 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:28.264080 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dd2v\" (UniqueName: \"kubernetes.io/projected/310b06ce-c5b4-4fb0-bb20-c663da8ebbf4-kube-api-access-4dd2v\") pod \"310b06ce-c5b4-4fb0-bb20-c663da8ebbf4\" (UID: \"310b06ce-c5b4-4fb0-bb20-c663da8ebbf4\") " Jul 24 13:53:28.264246 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:28.264193 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/310b06ce-c5b4-4fb0-bb20-c663da8ebbf4-extensions-socket-volume\") pod \"310b06ce-c5b4-4fb0-bb20-c663da8ebbf4\" (UID: \"310b06ce-c5b4-4fb0-bb20-c663da8ebbf4\") " Jul 24 13:53:28.264571 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:28.264543 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/310b06ce-c5b4-4fb0-bb20-c663da8ebbf4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "310b06ce-c5b4-4fb0-bb20-c663da8ebbf4" (UID: "310b06ce-c5b4-4fb0-bb20-c663da8ebbf4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:53:28.266819 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:28.266768 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/310b06ce-c5b4-4fb0-bb20-c663da8ebbf4-kube-api-access-4dd2v" (OuterVolumeSpecName: "kube-api-access-4dd2v") pod "310b06ce-c5b4-4fb0-bb20-c663da8ebbf4" (UID: "310b06ce-c5b4-4fb0-bb20-c663da8ebbf4"). InnerVolumeSpecName "kube-api-access-4dd2v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:53:28.365155 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:28.365122 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dd2v\" (UniqueName: \"kubernetes.io/projected/310b06ce-c5b4-4fb0-bb20-c663da8ebbf4-kube-api-access-4dd2v\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:53:28.365155 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:28.365150 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/310b06ce-c5b4-4fb0-bb20-c663da8ebbf4-extensions-socket-volume\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 13:53:28.443774 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:28.443736 2579 generic.go:356] "Generic (PLEG): container finished" podID="310b06ce-c5b4-4fb0-bb20-c663da8ebbf4" containerID="425c094646840d4a5375c5a3c52ae20bc7e86de0860aa87b4417986a942c2dc0" exitCode=0 Jul 24 13:53:28.443946 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:28.443799 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b" Jul 24 13:53:28.443946 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:28.443832 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b" event={"ID":"310b06ce-c5b4-4fb0-bb20-c663da8ebbf4","Type":"ContainerDied","Data":"425c094646840d4a5375c5a3c52ae20bc7e86de0860aa87b4417986a942c2dc0"} Jul 24 13:53:28.443946 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:28.443893 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b" event={"ID":"310b06ce-c5b4-4fb0-bb20-c663da8ebbf4","Type":"ContainerDied","Data":"71e3b908e793fafd703df5483f9b1cfea622eac9d62d60b171b4cfc3791ba1ac"} Jul 24 13:53:28.443946 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:28.443917 2579 scope.go:117] "RemoveContainer" containerID="425c094646840d4a5375c5a3c52ae20bc7e86de0860aa87b4417986a942c2dc0" Jul 24 13:53:28.454646 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:28.454627 2579 scope.go:117] "RemoveContainer" containerID="425c094646840d4a5375c5a3c52ae20bc7e86de0860aa87b4417986a942c2dc0" Jul 24 13:53:28.454941 ip-10-0-137-60 kubenswrapper[2579]: E0724 13:53:28.454911 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"425c094646840d4a5375c5a3c52ae20bc7e86de0860aa87b4417986a942c2dc0\": container with ID starting with 425c094646840d4a5375c5a3c52ae20bc7e86de0860aa87b4417986a942c2dc0 not found: ID does not exist" containerID="425c094646840d4a5375c5a3c52ae20bc7e86de0860aa87b4417986a942c2dc0" Jul 24 13:53:28.455065 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:28.454940 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"425c094646840d4a5375c5a3c52ae20bc7e86de0860aa87b4417986a942c2dc0"} err="failed to get container status \"425c094646840d4a5375c5a3c52ae20bc7e86de0860aa87b4417986a942c2dc0\": rpc error: code = NotFound desc = could not find container \"425c094646840d4a5375c5a3c52ae20bc7e86de0860aa87b4417986a942c2dc0\": container with ID starting with 425c094646840d4a5375c5a3c52ae20bc7e86de0860aa87b4417986a942c2dc0 not found: ID does not exist" Jul 24 13:53:28.467307 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:28.467277 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b"] Jul 24 13:53:28.472443 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:28.472418 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7664b"] Jul 24 13:53:30.420049 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:53:30.420016 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="310b06ce-c5b4-4fb0-bb20-c663da8ebbf4" path="/var/lib/kubelet/pods/310b06ce-c5b4-4fb0-bb20-c663da8ebbf4/volumes" Jul 24 13:54:08.385883 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:08.385847 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:26.685612 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:26.685570 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r69tn"] Jul 24 13:54:26.686112 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:26.685936 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="310b06ce-c5b4-4fb0-bb20-c663da8ebbf4" containerName="manager" Jul 24 13:54:26.686112 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:26.685948 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="310b06ce-c5b4-4fb0-bb20-c663da8ebbf4" containerName="manager" Jul 24 13:54:26.686112 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:26.686033 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="310b06ce-c5b4-4fb0-bb20-c663da8ebbf4" containerName="manager" Jul 24 13:54:26.690328 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:26.690312 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r69tn" Jul 24 13:54:26.694008 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:26.693968 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-p287l\"" Jul 24 13:54:26.721103 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:26.721065 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r69tn"] Jul 24 13:54:26.783374 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:26.783341 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d33852d-3cbc-4ab4-9eef-ffaad1841a87-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-r69tn\" (UID: \"7d33852d-3cbc-4ab4-9eef-ffaad1841a87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r69tn" Jul 24 13:54:26.783565 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:26.783388 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qdvp\" (UniqueName: \"kubernetes.io/projected/7d33852d-3cbc-4ab4-9eef-ffaad1841a87-kube-api-access-9qdvp\") pod \"kuadrant-operator-controller-manager-86878c769f-r69tn\" (UID: \"7d33852d-3cbc-4ab4-9eef-ffaad1841a87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r69tn" Jul 24 13:54:26.884400 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:26.884357 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d33852d-3cbc-4ab4-9eef-ffaad1841a87-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-r69tn\" (UID: \"7d33852d-3cbc-4ab4-9eef-ffaad1841a87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r69tn" Jul 24 13:54:26.884588 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:26.884408 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qdvp\" (UniqueName: \"kubernetes.io/projected/7d33852d-3cbc-4ab4-9eef-ffaad1841a87-kube-api-access-9qdvp\") pod \"kuadrant-operator-controller-manager-86878c769f-r69tn\" (UID: \"7d33852d-3cbc-4ab4-9eef-ffaad1841a87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r69tn" Jul 24 13:54:26.884772 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:26.884752 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d33852d-3cbc-4ab4-9eef-ffaad1841a87-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-r69tn\" (UID: \"7d33852d-3cbc-4ab4-9eef-ffaad1841a87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r69tn" Jul 24 13:54:26.897313 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:26.897283 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qdvp\" (UniqueName: \"kubernetes.io/projected/7d33852d-3cbc-4ab4-9eef-ffaad1841a87-kube-api-access-9qdvp\") pod \"kuadrant-operator-controller-manager-86878c769f-r69tn\" (UID: \"7d33852d-3cbc-4ab4-9eef-ffaad1841a87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r69tn" Jul 24 13:54:27.000453 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:27.000353 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r69tn" Jul 24 13:54:27.151030 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:27.150985 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r69tn"] Jul 24 13:54:27.152676 ip-10-0-137-60 kubenswrapper[2579]: W0724 13:54:27.152650 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d33852d_3cbc_4ab4_9eef_ffaad1841a87.slice/crio-b57a16292a74c3ae9e490bd209824fb01efc3a04ff4fe12323da56882f51ede0 WatchSource:0}: Error finding container b57a16292a74c3ae9e490bd209824fb01efc3a04ff4fe12323da56882f51ede0: Status 404 returned error can't find the container with id b57a16292a74c3ae9e490bd209824fb01efc3a04ff4fe12323da56882f51ede0 Jul 24 13:54:27.754776 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:27.754736 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r69tn" event={"ID":"7d33852d-3cbc-4ab4-9eef-ffaad1841a87","Type":"ContainerStarted","Data":"904e000d6371b657c757bd6235fc0e1c735c461d95de2a59d89ccbcde881b860"} Jul 24 13:54:27.755194 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:27.754790 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r69tn" event={"ID":"7d33852d-3cbc-4ab4-9eef-ffaad1841a87","Type":"ContainerStarted","Data":"b57a16292a74c3ae9e490bd209824fb01efc3a04ff4fe12323da56882f51ede0"} Jul 24 13:54:27.755194 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:27.754904 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r69tn" Jul 24 13:54:27.802494 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:27.802426 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r69tn" podStartSLOduration=1.802410896 podStartE2EDuration="1.802410896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:54:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:54:27.800020155 +0000 UTC m=+1209.923402547" watchObservedRunningTime="2026-07-24 13:54:27.802410896 +0000 UTC m=+1209.925793294" Jul 24 13:54:38.763952 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:38.763916 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r69tn" Jul 24 13:54:48.413787 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:48.413747 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:54:59.920638 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:54:59.920590 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:55:10.811292 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:55:10.811212 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:55:13.734939 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:55:13.734897 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:13.847858 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:55:13.847822 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:21.658361 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:55:21.658323 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:55:33.723705 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:55:33.723666 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:55:43.470725 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:55:43.470690 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:55:52.853012 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:55:52.852950 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:56:03.240419 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:56:03.240379 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:56:14.059757 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:56:14.059717 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:56:23.427511 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:56:23.427477 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:56:34.616647 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:56:34.616607 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:56:56.020229 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:56:56.020142 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:57:06.070111 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:57:06.070069 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:57:06.810068 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:57:06.810029 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:09.807870 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:57:09.807832 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:21.913238 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:57:21.913206 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:57:25.882469 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:57:25.882428 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:27.085320 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:57:27.085287 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:31.809405 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:57:31.809360 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:57:43.305320 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:57:43.305281 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:57:52.127625 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:57:52.127587 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:58:05.523114 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:58:05.523073 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:58:19.458566 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:58:19.458526 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:58:28.612420 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:58:28.612385 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:58:42.370303 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:58:42.370269 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:58:51.872130 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:58:51.872092 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:59:05.516437 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:59:05.516397 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:59:14.945596 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:59:14.945554 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:59:25.064426 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:59:25.064388 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:59:38.545563 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:59:38.545527 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:59:50.801748 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:59:50.801717 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 13:59:58.918740 ip-10-0-137-60 kubenswrapper[2579]: I0724 13:59:58.918704 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:00:00.153527 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:00.153490 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748360-ljc7k"] Jul 24 14:00:00.157208 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:00.157189 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748360-ljc7k" Jul 24 14:00:00.161199 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:00.161180 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4stw\"" Jul 24 14:00:00.167184 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:00.167168 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748360-ljc7k" Jul 24 14:00:00.178587 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:00.178563 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748360-ljc7k"] Jul 24 14:00:00.312155 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:00.312133 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748360-ljc7k"] Jul 24 14:00:00.314098 ip-10-0-137-60 kubenswrapper[2579]: W0724 14:00:00.314062 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ef16b8e_45e6_48d6_9940_361a9184a0ce.slice/crio-21400b1990b2eeca4d5db1543404f9bec5a1c5b1a3afae70dc2b7d2f88c83f96 WatchSource:0}: Error finding container 21400b1990b2eeca4d5db1543404f9bec5a1c5b1a3afae70dc2b7d2f88c83f96: Status 404 returned error can't find the container with id 21400b1990b2eeca4d5db1543404f9bec5a1c5b1a3afae70dc2b7d2f88c83f96 Jul 24 14:00:00.564271 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:00.564234 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748360-ljc7k" event={"ID":"7ef16b8e-45e6-48d6-9940-361a9184a0ce","Type":"ContainerStarted","Data":"21400b1990b2eeca4d5db1543404f9bec5a1c5b1a3afae70dc2b7d2f88c83f96"} Jul 24 14:00:01.571438 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:01.571402 2579 generic.go:356] "Generic (PLEG): container finished" podID="7ef16b8e-45e6-48d6-9940-361a9184a0ce" containerID="07f39498b301b1f57e6ac07b288321e66cf4fc1fee3cfc56cdac8e56ac1e95af" exitCode=0 Jul 24 14:00:01.571830 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:01.571469 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748360-ljc7k" event={"ID":"7ef16b8e-45e6-48d6-9940-361a9184a0ce","Type":"ContainerDied","Data":"07f39498b301b1f57e6ac07b288321e66cf4fc1fee3cfc56cdac8e56ac1e95af"} Jul 24 14:00:02.712493 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:02.712466 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748360-ljc7k" Jul 24 14:00:03.582803 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:03.582774 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748360-ljc7k" Jul 24 14:00:03.582803 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:03.582790 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748360-ljc7k" event={"ID":"7ef16b8e-45e6-48d6-9940-361a9184a0ce","Type":"ContainerDied","Data":"21400b1990b2eeca4d5db1543404f9bec5a1c5b1a3afae70dc2b7d2f88c83f96"} Jul 24 14:00:03.583033 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:03.582839 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21400b1990b2eeca4d5db1543404f9bec5a1c5b1a3afae70dc2b7d2f88c83f96" Jul 24 14:00:10.720235 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:10.720189 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:00:23.324615 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:23.324574 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:00:35.116529 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:35.116495 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:00:38.956436 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:38.956401 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:00:44.316428 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:44.316388 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:00:46.392609 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:46.392571 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:00:57.516625 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:00:57.516589 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:01:02.125763 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:02.125725 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:05.377227 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:05.377195 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:05.488921 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:05.488884 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:05.967638 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:05.967603 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:06.614868 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:06.614829 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:01:12.360189 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:12.360104 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:12.461318 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:12.461286 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:12.652061 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:12.651932 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:13.032967 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.032878 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6"] Jul 24 14:01:13.033471 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.033453 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ef16b8e-45e6-48d6-9940-361a9184a0ce" containerName="cleanup" Jul 24 14:01:13.033516 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.033478 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ef16b8e-45e6-48d6-9940-361a9184a0ce" containerName="cleanup" Jul 24 14:01:13.033618 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.033605 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ef16b8e-45e6-48d6-9940-361a9184a0ce" containerName="cleanup" Jul 24 14:01:13.037286 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.037265 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.041035 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.041013 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4dc6ea7f-openshift-default-dockercfg-bmxnh\"" Jul 24 14:01:13.058894 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.058866 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6"] Jul 24 14:01:13.108006 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.107957 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:13.197606 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.197574 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:13.225077 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.225044 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-data\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.225271 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.225168 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-envoy\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.225271 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.225253 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-workload-certs\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.225400 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.225286 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-token\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.225400 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.225315 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-podinfo\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.225400 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.225352 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-workload-socket\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.225400 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.225386 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcvzk\" (UniqueName: \"kubernetes.io/projected/ad4f4258-65e7-44b1-88e5-3da42b8ead96-kube-api-access-mcvzk\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.225599 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.225422 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istiod-ca-cert\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.225599 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.225455 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-credential-socket\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.326431 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.326400 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-data\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.326628 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.326437 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-envoy\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.326628 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.326479 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-workload-certs\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.326628 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.326595 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-token\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.326789 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.326641 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-podinfo\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.326789 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.326676 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-workload-socket\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.326789 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.326704 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcvzk\" (UniqueName: \"kubernetes.io/projected/ad4f4258-65e7-44b1-88e5-3da42b8ead96-kube-api-access-mcvzk\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.326789 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.326738 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istiod-ca-cert\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.326789 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.326768 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-credential-socket\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.326982 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.326949 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-workload-certs\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.327148 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.327122 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-credential-socket\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.327325 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.327194 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-data\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.327325 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.327231 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-workload-socket\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.327952 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.327916 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istiod-ca-cert\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.329243 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.329222 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-envoy\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.329437 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.329420 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-podinfo\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.336241 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.336217 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-token\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.336330 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.336236 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcvzk\" (UniqueName: \"kubernetes.io/projected/ad4f4258-65e7-44b1-88e5-3da42b8ead96-kube-api-access-mcvzk\") pod \"e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.348698 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.348674 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:13.499692 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.499661 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6"] Jul 24 14:01:13.500752 ip-10-0-137-60 kubenswrapper[2579]: W0724 14:01:13.500726 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad4f4258_65e7_44b1_88e5_3da42b8ead96.slice/crio-d73fb14a4eac35d5fe84d77c3b598f555d94e6d16db2da25b60e5712f2d56aa5 WatchSource:0}: Error finding container d73fb14a4eac35d5fe84d77c3b598f555d94e6d16db2da25b60e5712f2d56aa5: Status 404 returned error can't find the container with id d73fb14a4eac35d5fe84d77c3b598f555d94e6d16db2da25b60e5712f2d56aa5 Jul 24 14:01:13.503082 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.503053 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:01:13.503159 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.503121 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:01:13.503159 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.503150 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:01:13.995326 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.995288 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" event={"ID":"ad4f4258-65e7-44b1-88e5-3da42b8ead96","Type":"ContainerStarted","Data":"f169629b38a7c4b108375819db49d1d84305172ec30757f2520ae8645b787f8f"} Jul 24 14:01:13.995513 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:13.995346 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" event={"ID":"ad4f4258-65e7-44b1-88e5-3da42b8ead96","Type":"ContainerStarted","Data":"d73fb14a4eac35d5fe84d77c3b598f555d94e6d16db2da25b60e5712f2d56aa5"} Jul 24 14:01:14.022666 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:14.022605 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" podStartSLOduration=2.022585579 podStartE2EDuration="2.022585579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:01:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:01:14.019883624 +0000 UTC m=+1616.143266029" watchObservedRunningTime="2026-07-24 14:01:14.022585579 +0000 UTC m=+1616.145967976" Jul 24 14:01:14.349855 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:14.349822 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:14.354599 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:14.354568 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:15.000434 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:15.000403 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:15.001291 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:15.001271 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:15.415134 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:15.415099 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:15.516008 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:15.515959 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:15.608489 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:15.608458 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:15.730698 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:15.730606 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:16.346592 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:16.346554 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:16.450272 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:16.450237 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:21.737438 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:21.737391 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:22.369140 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:22.369095 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:01:30.231744 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:30.231696 2579 patch_prober.go:28] interesting pod/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": read tcp 10.133.0.2:41462->10.133.0.45:15021: read: connection reset by peer" start-of-body= Jul 24 14:01:30.232174 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:30.231785 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" podUID="ad4f4258-65e7-44b1-88e5-3da42b8ead96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": read tcp 10.133.0.2:41462->10.133.0.45:15021: read: connection reset by peer" Jul 24 14:01:31.102436 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:31.102402 2579 generic.go:356] "Generic (PLEG): container finished" podID="ad4f4258-65e7-44b1-88e5-3da42b8ead96" containerID="f169629b38a7c4b108375819db49d1d84305172ec30757f2520ae8645b787f8f" exitCode=0 Jul 24 14:01:31.102620 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:31.102481 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" event={"ID":"ad4f4258-65e7-44b1-88e5-3da42b8ead96","Type":"ContainerDied","Data":"f169629b38a7c4b108375819db49d1d84305172ec30757f2520ae8645b787f8f"} Jul 24 14:01:31.102958 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:31.102937 2579 scope.go:117] "RemoveContainer" containerID="f169629b38a7c4b108375819db49d1d84305172ec30757f2520ae8645b787f8f" Jul 24 14:01:31.104036 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:31.104005 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:01:31.104132 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:31.104094 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:01:31.104172 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:31.104133 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:01:32.109758 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:32.109722 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" event={"ID":"ad4f4258-65e7-44b1-88e5-3da42b8ead96","Type":"ContainerStarted","Data":"0fe48f63c523e3b8543f42637c3bc2af3a4ea59c1c0141566549a0b8bcd198b0"} Jul 24 14:01:32.349896 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:32.349857 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:32.350572 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:32.350207 2579 patch_prober.go:28] interesting pod/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 24 14:01:32.350572 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:32.350290 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" podUID="ad4f4258-65e7-44b1-88e5-3da42b8ead96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 24 14:01:33.349771 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:33.349725 2579 patch_prober.go:28] interesting pod/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 24 14:01:33.350315 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:33.349804 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" podUID="ad4f4258-65e7-44b1-88e5-3da42b8ead96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 24 14:01:34.350115 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:34.350071 2579 patch_prober.go:28] interesting pod/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 24 14:01:34.350713 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:34.350677 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" podUID="ad4f4258-65e7-44b1-88e5-3da42b8ead96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 24 14:01:35.350189 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:35.350150 2579 patch_prober.go:28] interesting pod/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 24 14:01:35.350823 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:35.350788 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" podUID="ad4f4258-65e7-44b1-88e5-3da42b8ead96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 24 14:01:36.353068 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:36.353037 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:36.353468 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:36.353302 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:36.354038 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:36.354020 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:01:51.933928 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:51.933893 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:52.032211 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:52.032175 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:52.134124 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:52.134077 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:54.939353 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:54.939320 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:01:56.171006 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:56.170951 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:56.331978 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:56.331941 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6"] Jul 24 14:01:56.332191 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:56.332168 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" podUID="ad4f4258-65e7-44b1-88e5-3da42b8ead96" containerName="istio-proxy" containerID="cri-o://0fe48f63c523e3b8543f42637c3bc2af3a4ea59c1c0141566549a0b8bcd198b0" gracePeriod=30 Jul 24 14:01:56.402719 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:56.402683 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:56.617269 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:56.617239 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:56.937438 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:56.937338 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:57.056887 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.056854 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:57.151062 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.151025 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:57.723932 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.723894 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht"] Jul 24 14:01:57.726596 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.726579 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.735731 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.735708 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-4a9d360c-gw-openshift-default-dockercfg-nmjkz\"" Jul 24 14:01:57.772251 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.772217 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht"] Jul 24 14:01:57.818198 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.818147 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:57.850594 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.850552 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-podinfo\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.850799 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.850600 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-workload-socket\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.850799 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.850651 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-data\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.850799 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.850740 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-credential-socket\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.850799 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.850766 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-token\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.850799 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.850789 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-workload-certs\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.851122 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.850845 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-envoy\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.851122 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.850877 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kldxr\" (UniqueName: \"kubernetes.io/projected/74b70144-14b0-4bdd-9952-6fae60c0ca34-kube-api-access-kldxr\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.851122 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.850935 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74b70144-14b0-4bdd-9952-6fae60c0ca34-istiod-ca-cert\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.952107 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.952061 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-envoy\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.952107 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.952100 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kldxr\" (UniqueName: \"kubernetes.io/projected/74b70144-14b0-4bdd-9952-6fae60c0ca34-kube-api-access-kldxr\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.952372 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.952122 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74b70144-14b0-4bdd-9952-6fae60c0ca34-istiod-ca-cert\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.952372 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.952189 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-podinfo\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.952372 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.952220 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-workload-socket\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.952372 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.952269 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-data\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.952372 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.952332 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-credential-socket\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.952372 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.952358 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-token\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.952660 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.952381 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-workload-certs\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.952803 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.952768 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-workload-certs\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.952878 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.952855 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-credential-socket\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.952934 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.952895 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-workload-socket\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.952984 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.952927 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-data\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.953516 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.953496 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74b70144-14b0-4bdd-9952-6fae60c0ca34-istiod-ca-cert\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.954980 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.954958 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-envoy\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.955164 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.955149 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-podinfo\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.964908 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.964884 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kldxr\" (UniqueName: \"kubernetes.io/projected/74b70144-14b0-4bdd-9952-6fae60c0ca34-kube-api-access-kldxr\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:57.965047 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:57.965030 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-token\") pod \"e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:58.036405 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:58.036312 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:01:58.114527 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:58.114498 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:58.195346 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:58.195322 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht"] Jul 24 14:01:58.197315 ip-10-0-137-60 kubenswrapper[2579]: W0724 14:01:58.197285 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b70144_14b0_4bdd_9952_6fae60c0ca34.slice/crio-598357c8628664b881a17596893ed9e13107af682edaffde11d98d93fdbd5af0 WatchSource:0}: Error finding container 598357c8628664b881a17596893ed9e13107af682edaffde11d98d93fdbd5af0: Status 404 returned error can't find the container with id 598357c8628664b881a17596893ed9e13107af682edaffde11d98d93fdbd5af0 Jul 24 14:01:58.199412 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:58.199382 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:01:58.199493 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:58.199449 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:01:58.199493 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:58.199478 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:01:58.253596 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:58.253555 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" event={"ID":"74b70144-14b0-4bdd-9952-6fae60c0ca34","Type":"ContainerStarted","Data":"598357c8628664b881a17596893ed9e13107af682edaffde11d98d93fdbd5af0"} Jul 24 14:01:59.260570 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:59.260531 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" event={"ID":"74b70144-14b0-4bdd-9952-6fae60c0ca34","Type":"ContainerStarted","Data":"bb80d92136a073f99c9df25d0d8e5e529912e8c3d7a675e2f60e397bdef029f6"} Jul 24 14:01:59.297917 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:01:59.297860 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" podStartSLOduration=2.297843552 podStartE2EDuration="2.297843552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:01:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:01:59.295681469 +0000 UTC m=+1661.419063868" watchObservedRunningTime="2026-07-24 14:01:59.297843552 +0000 UTC m=+1661.421225951" Jul 24 14:02:00.037188 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:00.037143 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:02:00.042479 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:00.042449 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:02:00.266244 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:00.266212 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:02:00.267377 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:00.267360 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:02:00.819612 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:00.819581 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:00.957149 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:00.957113 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:01.058498 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.058458 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:01.539906 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.539870 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:01.609179 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.609151 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:02:01.640460 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.640429 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:01.690013 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.689951 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-workload-certs\") pod \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " Jul 24 14:02:01.690013 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.689986 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-data\") pod \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " Jul 24 14:02:01.690261 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.690030 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcvzk\" (UniqueName: \"kubernetes.io/projected/ad4f4258-65e7-44b1-88e5-3da42b8ead96-kube-api-access-mcvzk\") pod \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " Jul 24 14:02:01.690261 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.690064 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-credential-socket\") pod \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " Jul 24 14:02:01.690261 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.690084 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-token\") pod \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " Jul 24 14:02:01.690261 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.690118 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-podinfo\") pod \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " Jul 24 14:02:01.690261 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.690200 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-workload-socket\") pod \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " Jul 24 14:02:01.690261 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.690234 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istiod-ca-cert\") pod \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " Jul 24 14:02:01.690261 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.690241 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ad4f4258-65e7-44b1-88e5-3da42b8ead96" (UID: "ad4f4258-65e7-44b1-88e5-3da42b8ead96"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:02:01.690620 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.690273 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-envoy\") pod \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\" (UID: \"ad4f4258-65e7-44b1-88e5-3da42b8ead96\") " Jul 24 14:02:01.690620 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.690429 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ad4f4258-65e7-44b1-88e5-3da42b8ead96" (UID: "ad4f4258-65e7-44b1-88e5-3da42b8ead96"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:02:01.690620 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.690526 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ad4f4258-65e7-44b1-88e5-3da42b8ead96" (UID: "ad4f4258-65e7-44b1-88e5-3da42b8ead96"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:02:01.690788 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.690624 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-workload-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:02:01.690788 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.690643 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-workload-certs\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:02:01.690788 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.690656 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-credential-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:02:01.690943 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.690848 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ad4f4258-65e7-44b1-88e5-3da42b8ead96" (UID: "ad4f4258-65e7-44b1-88e5-3da42b8ead96"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:02:01.691028 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.690934 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-data" (OuterVolumeSpecName: "istio-data") pod "ad4f4258-65e7-44b1-88e5-3da42b8ead96" (UID: "ad4f4258-65e7-44b1-88e5-3da42b8ead96"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:02:01.692543 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.692507 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad4f4258-65e7-44b1-88e5-3da42b8ead96-kube-api-access-mcvzk" (OuterVolumeSpecName: "kube-api-access-mcvzk") pod "ad4f4258-65e7-44b1-88e5-3da42b8ead96" (UID: "ad4f4258-65e7-44b1-88e5-3da42b8ead96"). InnerVolumeSpecName "kube-api-access-mcvzk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:02:01.692660 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.692607 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ad4f4258-65e7-44b1-88e5-3da42b8ead96" (UID: "ad4f4258-65e7-44b1-88e5-3da42b8ead96"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:02:01.692713 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.692693 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-token" (OuterVolumeSpecName: "istio-token") pod "ad4f4258-65e7-44b1-88e5-3da42b8ead96" (UID: "ad4f4258-65e7-44b1-88e5-3da42b8ead96"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:02:01.692894 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.692878 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ad4f4258-65e7-44b1-88e5-3da42b8ead96" (UID: "ad4f4258-65e7-44b1-88e5-3da42b8ead96"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:02:01.791967 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.791930 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istiod-ca-cert\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:02:01.791967 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.791961 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-envoy\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:02:01.791967 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.791969 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-data\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:02:01.792214 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.791977 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcvzk\" (UniqueName: \"kubernetes.io/projected/ad4f4258-65e7-44b1-88e5-3da42b8ead96-kube-api-access-mcvzk\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:02:01.792214 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.792013 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-token\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:02:01.792214 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:01.792022 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad4f4258-65e7-44b1-88e5-3da42b8ead96-istio-podinfo\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:02:02.279155 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:02.279123 2579 generic.go:356] "Generic (PLEG): container finished" podID="ad4f4258-65e7-44b1-88e5-3da42b8ead96" containerID="0fe48f63c523e3b8543f42637c3bc2af3a4ea59c1c0141566549a0b8bcd198b0" exitCode=0 Jul 24 14:02:02.279329 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:02.279174 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" Jul 24 14:02:02.279329 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:02.279213 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" event={"ID":"ad4f4258-65e7-44b1-88e5-3da42b8ead96","Type":"ContainerDied","Data":"0fe48f63c523e3b8543f42637c3bc2af3a4ea59c1c0141566549a0b8bcd198b0"} Jul 24 14:02:02.279329 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:02.279277 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6" event={"ID":"ad4f4258-65e7-44b1-88e5-3da42b8ead96","Type":"ContainerDied","Data":"d73fb14a4eac35d5fe84d77c3b598f555d94e6d16db2da25b60e5712f2d56aa5"} Jul 24 14:02:02.279329 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:02.279306 2579 scope.go:117] "RemoveContainer" containerID="0fe48f63c523e3b8543f42637c3bc2af3a4ea59c1c0141566549a0b8bcd198b0" Jul 24 14:02:02.289431 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:02.289407 2579 scope.go:117] "RemoveContainer" containerID="f169629b38a7c4b108375819db49d1d84305172ec30757f2520ae8645b787f8f" Jul 24 14:02:02.300600 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:02.300579 2579 scope.go:117] "RemoveContainer" containerID="0fe48f63c523e3b8543f42637c3bc2af3a4ea59c1c0141566549a0b8bcd198b0" Jul 24 14:02:02.300898 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:02:02.300862 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fe48f63c523e3b8543f42637c3bc2af3a4ea59c1c0141566549a0b8bcd198b0\": container with ID starting with 0fe48f63c523e3b8543f42637c3bc2af3a4ea59c1c0141566549a0b8bcd198b0 not found: ID does not exist" containerID="0fe48f63c523e3b8543f42637c3bc2af3a4ea59c1c0141566549a0b8bcd198b0" Jul 24 14:02:02.301028 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:02.300903 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fe48f63c523e3b8543f42637c3bc2af3a4ea59c1c0141566549a0b8bcd198b0"} err="failed to get container status \"0fe48f63c523e3b8543f42637c3bc2af3a4ea59c1c0141566549a0b8bcd198b0\": rpc error: code = NotFound desc = could not find container \"0fe48f63c523e3b8543f42637c3bc2af3a4ea59c1c0141566549a0b8bcd198b0\": container with ID starting with 0fe48f63c523e3b8543f42637c3bc2af3a4ea59c1c0141566549a0b8bcd198b0 not found: ID does not exist" Jul 24 14:02:02.301028 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:02.300920 2579 scope.go:117] "RemoveContainer" containerID="f169629b38a7c4b108375819db49d1d84305172ec30757f2520ae8645b787f8f" Jul 24 14:02:02.301269 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:02:02.301243 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f169629b38a7c4b108375819db49d1d84305172ec30757f2520ae8645b787f8f\": container with ID starting with f169629b38a7c4b108375819db49d1d84305172ec30757f2520ae8645b787f8f not found: ID does not exist" containerID="f169629b38a7c4b108375819db49d1d84305172ec30757f2520ae8645b787f8f" Jul 24 14:02:02.301311 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:02.301280 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f169629b38a7c4b108375819db49d1d84305172ec30757f2520ae8645b787f8f"} err="failed to get container status \"f169629b38a7c4b108375819db49d1d84305172ec30757f2520ae8645b787f8f\": rpc error: code = NotFound desc = could not find container \"f169629b38a7c4b108375819db49d1d84305172ec30757f2520ae8645b787f8f\": container with ID starting with f169629b38a7c4b108375819db49d1d84305172ec30757f2520ae8645b787f8f not found: ID does not exist" Jul 24 14:02:02.325268 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:02.325233 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6"] Jul 24 14:02:02.331114 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:02.331087 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4dc6ea7f-openshift-default-95fff46d8-7pxv6"] Jul 24 14:02:02.420268 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:02.420233 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad4f4258-65e7-44b1-88e5-3da42b8ead96" path="/var/lib/kubelet/pods/ad4f4258-65e7-44b1-88e5-3da42b8ead96/volumes" Jul 24 14:02:06.021200 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:06.021159 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:02:11.828180 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:11.828139 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:14.348768 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:14.348728 2579 generic.go:356] "Generic (PLEG): container finished" podID="74b70144-14b0-4bdd-9952-6fae60c0ca34" containerID="bb80d92136a073f99c9df25d0d8e5e529912e8c3d7a675e2f60e397bdef029f6" exitCode=0 Jul 24 14:02:14.349526 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:14.348815 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" event={"ID":"74b70144-14b0-4bdd-9952-6fae60c0ca34","Type":"ContainerDied","Data":"bb80d92136a073f99c9df25d0d8e5e529912e8c3d7a675e2f60e397bdef029f6"} Jul 24 14:02:14.349526 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:14.349385 2579 scope.go:117] "RemoveContainer" containerID="bb80d92136a073f99c9df25d0d8e5e529912e8c3d7a675e2f60e397bdef029f6" Jul 24 14:02:14.350508 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:14.350479 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:02:14.350566 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:14.350550 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:02:14.350605 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:14.350582 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:02:15.267010 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:15.266952 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:02:15.360694 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:15.360643 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" event={"ID":"74b70144-14b0-4bdd-9952-6fae60c0ca34","Type":"ContainerStarted","Data":"6c445e4a1f687cdca52d62b73cd362de07a338632b6ec53d156d06756ea16bd0"} Jul 24 14:02:16.036854 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:16.036810 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:02:16.038099 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:16.038076 2579 patch_prober.go:28] interesting pod/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 24 14:02:16.038221 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:16.038125 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" podUID="74b70144-14b0-4bdd-9952-6fae60c0ca34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 24 14:02:17.036938 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:17.036903 2579 patch_prober.go:28] interesting pod/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 24 14:02:17.037369 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:17.036958 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" podUID="74b70144-14b0-4bdd-9952-6fae60c0ca34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 24 14:02:18.037066 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:18.037022 2579 patch_prober.go:28] interesting pod/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 24 14:02:18.037467 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:18.037089 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" podUID="74b70144-14b0-4bdd-9952-6fae60c0ca34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 24 14:02:19.037804 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:19.037758 2579 patch_prober.go:28] interesting pod/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 24 14:02:19.038243 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:19.037836 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" podUID="74b70144-14b0-4bdd-9952-6fae60c0ca34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 24 14:02:20.041010 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:20.040962 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:02:20.041392 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:20.041345 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:02:20.042156 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:20.042135 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:02:36.730358 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:36.730323 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:36.830558 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:36.830520 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:36.929773 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:36.929736 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:39.805962 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:39.805884 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:02:40.994812 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:40.994774 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:41.176418 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:41.176389 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht"] Jul 24 14:02:41.176607 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:41.176583 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" podUID="74b70144-14b0-4bdd-9952-6fae60c0ca34" containerName="istio-proxy" containerID="cri-o://6c445e4a1f687cdca52d62b73cd362de07a338632b6ec53d156d06756ea16bd0" gracePeriod=30 Jul 24 14:02:41.197162 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:41.197131 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:41.430547 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:41.430510 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:41.677850 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:41.677816 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:42.016451 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.016407 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg"] Jul 24 14:02:42.018261 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.018231 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad4f4258-65e7-44b1-88e5-3da42b8ead96" containerName="istio-proxy" Jul 24 14:02:42.018499 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.018487 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad4f4258-65e7-44b1-88e5-3da42b8ead96" containerName="istio-proxy" Jul 24 14:02:42.018935 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.018907 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad4f4258-65e7-44b1-88e5-3da42b8ead96" containerName="istio-proxy" Jul 24 14:02:42.023084 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.023058 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.026546 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.026527 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6538ff29-openshift-default-dockercfg-qkvsj\"" Jul 24 14:02:42.042376 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.042346 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg"] Jul 24 14:02:42.058803 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.058765 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:42.086520 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.086485 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/835e3042-b9ad-4ff9-9462-4632d626b702-istiod-ca-cert\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.086520 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.086521 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-istio-data\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.086731 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.086542 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-workload-socket\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.086731 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.086563 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl87h\" (UniqueName: \"kubernetes.io/projected/835e3042-b9ad-4ff9-9462-4632d626b702-kube-api-access-jl87h\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.086731 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.086684 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-istio-envoy\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.086878 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.086730 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-workload-certs\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.086878 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.086810 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-credential-socket\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.087025 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.086901 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/835e3042-b9ad-4ff9-9462-4632d626b702-istio-podinfo\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.087025 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.086929 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/835e3042-b9ad-4ff9-9462-4632d626b702-istio-token\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.188041 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.187972 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-istio-data\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.188237 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.188049 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-workload-socket\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.188237 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.188079 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl87h\" (UniqueName: \"kubernetes.io/projected/835e3042-b9ad-4ff9-9462-4632d626b702-kube-api-access-jl87h\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.188237 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.188127 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-istio-envoy\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.188237 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.188172 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-workload-certs\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.188237 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.188204 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-credential-socket\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.188513 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.188280 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/835e3042-b9ad-4ff9-9462-4632d626b702-istio-podinfo\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.188513 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.188304 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/835e3042-b9ad-4ff9-9462-4632d626b702-istio-token\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.188513 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.188369 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/835e3042-b9ad-4ff9-9462-4632d626b702-istiod-ca-cert\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.188513 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.188462 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-istio-data\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.188513 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.188477 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-workload-socket\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.188694 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.188522 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-workload-certs\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.188869 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.188846 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-credential-socket\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.188956 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.188942 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/835e3042-b9ad-4ff9-9462-4632d626b702-istiod-ca-cert\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.190616 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.190590 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-istio-envoy\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.190812 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.190793 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/835e3042-b9ad-4ff9-9462-4632d626b702-istio-podinfo\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.203095 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.203017 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/835e3042-b9ad-4ff9-9462-4632d626b702-istio-token\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.204725 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.204703 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl87h\" (UniqueName: \"kubernetes.io/projected/835e3042-b9ad-4ff9-9462-4632d626b702-kube-api-access-jl87h\") pod \"e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.333545 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.333507 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:42.360391 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.360360 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:42.488091 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.488064 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg"] Jul 24 14:02:42.489393 ip-10-0-137-60 kubenswrapper[2579]: W0724 14:02:42.489339 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod835e3042_b9ad_4ff9_9462_4632d626b702.slice/crio-9c5bb88fca6ae47a93631310b2d8f78f0583b16965c7d09bc6c745ce77db60ec WatchSource:0}: Error finding container 9c5bb88fca6ae47a93631310b2d8f78f0583b16965c7d09bc6c745ce77db60ec: Status 404 returned error can't find the container with id 9c5bb88fca6ae47a93631310b2d8f78f0583b16965c7d09bc6c745ce77db60ec Jul 24 14:02:42.492190 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.492155 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:02:42.492281 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.492220 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:02:42.492281 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.492259 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:02:42.518532 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:42.518503 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" event={"ID":"835e3042-b9ad-4ff9-9462-4632d626b702","Type":"ContainerStarted","Data":"9c5bb88fca6ae47a93631310b2d8f78f0583b16965c7d09bc6c745ce77db60ec"} Jul 24 14:02:43.525824 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:43.525785 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" event={"ID":"835e3042-b9ad-4ff9-9462-4632d626b702","Type":"ContainerStarted","Data":"7566d1a6b3cdaad1616a834b6c8bbb12c79346f12e08be57efe3b539644af2d5"} Jul 24 14:02:43.552796 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:43.552722 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" podStartSLOduration=2.552702127 podStartE2EDuration="2.552702127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:02:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:02:43.549549376 +0000 UTC m=+1705.672931787" watchObservedRunningTime="2026-07-24 14:02:43.552702127 +0000 UTC m=+1705.676084524" Jul 24 14:02:44.333736 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:44.333702 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:44.338532 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:44.338506 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:44.531719 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:44.531684 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:44.532864 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:44.532842 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:02:44.964950 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:44.964918 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:45.230061 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:45.229965 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:45.427137 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:45.427103 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:46.384679 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.384639 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:46.434698 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.434673 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:02:46.532847 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.532751 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-workload-certs\") pod \"74b70144-14b0-4bdd-9952-6fae60c0ca34\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " Jul 24 14:02:46.532847 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.532801 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-credential-socket\") pod \"74b70144-14b0-4bdd-9952-6fae60c0ca34\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " Jul 24 14:02:46.532847 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.532841 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74b70144-14b0-4bdd-9952-6fae60c0ca34-istiod-ca-cert\") pod \"74b70144-14b0-4bdd-9952-6fae60c0ca34\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " Jul 24 14:02:46.533176 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.532883 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-data\") pod \"74b70144-14b0-4bdd-9952-6fae60c0ca34\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " Jul 24 14:02:46.533176 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.532911 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-token\") pod \"74b70144-14b0-4bdd-9952-6fae60c0ca34\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " Jul 24 14:02:46.533176 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.532946 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kldxr\" (UniqueName: \"kubernetes.io/projected/74b70144-14b0-4bdd-9952-6fae60c0ca34-kube-api-access-kldxr\") pod \"74b70144-14b0-4bdd-9952-6fae60c0ca34\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " Jul 24 14:02:46.533176 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.533016 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-envoy\") pod \"74b70144-14b0-4bdd-9952-6fae60c0ca34\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " Jul 24 14:02:46.533176 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.533052 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-podinfo\") pod \"74b70144-14b0-4bdd-9952-6fae60c0ca34\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " Jul 24 14:02:46.533176 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.533085 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-workload-socket\") pod \"74b70144-14b0-4bdd-9952-6fae60c0ca34\" (UID: \"74b70144-14b0-4bdd-9952-6fae60c0ca34\") " Jul 24 14:02:46.533176 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.533098 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "74b70144-14b0-4bdd-9952-6fae60c0ca34" (UID: "74b70144-14b0-4bdd-9952-6fae60c0ca34"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:02:46.533508 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.533239 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "74b70144-14b0-4bdd-9952-6fae60c0ca34" (UID: "74b70144-14b0-4bdd-9952-6fae60c0ca34"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:02:46.533508 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.533367 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74b70144-14b0-4bdd-9952-6fae60c0ca34-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "74b70144-14b0-4bdd-9952-6fae60c0ca34" (UID: "74b70144-14b0-4bdd-9952-6fae60c0ca34"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:02:46.533508 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.533482 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "74b70144-14b0-4bdd-9952-6fae60c0ca34" (UID: "74b70144-14b0-4bdd-9952-6fae60c0ca34"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:02:46.533730 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.533653 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-credential-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:02:46.533730 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.533675 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74b70144-14b0-4bdd-9952-6fae60c0ca34-istiod-ca-cert\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:02:46.533730 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.533689 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-workload-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:02:46.533730 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.533701 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-workload-certs\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:02:46.533939 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.533832 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-data" (OuterVolumeSpecName: "istio-data") pod "74b70144-14b0-4bdd-9952-6fae60c0ca34" (UID: "74b70144-14b0-4bdd-9952-6fae60c0ca34"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:02:46.535534 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.535479 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "74b70144-14b0-4bdd-9952-6fae60c0ca34" (UID: "74b70144-14b0-4bdd-9952-6fae60c0ca34"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:02:46.535682 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.535657 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "74b70144-14b0-4bdd-9952-6fae60c0ca34" (UID: "74b70144-14b0-4bdd-9952-6fae60c0ca34"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:02:46.536316 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.536273 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74b70144-14b0-4bdd-9952-6fae60c0ca34-kube-api-access-kldxr" (OuterVolumeSpecName: "kube-api-access-kldxr") pod "74b70144-14b0-4bdd-9952-6fae60c0ca34" (UID: "74b70144-14b0-4bdd-9952-6fae60c0ca34"). InnerVolumeSpecName "kube-api-access-kldxr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:02:46.536316 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.536312 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-token" (OuterVolumeSpecName: "istio-token") pod "74b70144-14b0-4bdd-9952-6fae60c0ca34" (UID: "74b70144-14b0-4bdd-9952-6fae60c0ca34"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:02:46.547189 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.546930 2579 generic.go:356] "Generic (PLEG): container finished" podID="74b70144-14b0-4bdd-9952-6fae60c0ca34" containerID="6c445e4a1f687cdca52d62b73cd362de07a338632b6ec53d156d06756ea16bd0" exitCode=0 Jul 24 14:02:46.547189 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.546966 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" event={"ID":"74b70144-14b0-4bdd-9952-6fae60c0ca34","Type":"ContainerDied","Data":"6c445e4a1f687cdca52d62b73cd362de07a338632b6ec53d156d06756ea16bd0"} Jul 24 14:02:46.547189 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.547054 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" event={"ID":"74b70144-14b0-4bdd-9952-6fae60c0ca34","Type":"ContainerDied","Data":"598357c8628664b881a17596893ed9e13107af682edaffde11d98d93fdbd5af0"} Jul 24 14:02:46.547189 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.547085 2579 scope.go:117] "RemoveContainer" containerID="6c445e4a1f687cdca52d62b73cd362de07a338632b6ec53d156d06756ea16bd0" Jul 24 14:02:46.547509 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.547462 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht" Jul 24 14:02:46.558820 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.558799 2579 scope.go:117] "RemoveContainer" containerID="bb80d92136a073f99c9df25d0d8e5e529912e8c3d7a675e2f60e397bdef029f6" Jul 24 14:02:46.569061 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.569039 2579 scope.go:117] "RemoveContainer" containerID="6c445e4a1f687cdca52d62b73cd362de07a338632b6ec53d156d06756ea16bd0" Jul 24 14:02:46.569342 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:02:46.569319 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c445e4a1f687cdca52d62b73cd362de07a338632b6ec53d156d06756ea16bd0\": container with ID starting with 6c445e4a1f687cdca52d62b73cd362de07a338632b6ec53d156d06756ea16bd0 not found: ID does not exist" containerID="6c445e4a1f687cdca52d62b73cd362de07a338632b6ec53d156d06756ea16bd0" Jul 24 14:02:46.569427 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.569350 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c445e4a1f687cdca52d62b73cd362de07a338632b6ec53d156d06756ea16bd0"} err="failed to get container status \"6c445e4a1f687cdca52d62b73cd362de07a338632b6ec53d156d06756ea16bd0\": rpc error: code = NotFound desc = could not find container \"6c445e4a1f687cdca52d62b73cd362de07a338632b6ec53d156d06756ea16bd0\": container with ID starting with 6c445e4a1f687cdca52d62b73cd362de07a338632b6ec53d156d06756ea16bd0 not found: ID does not exist" Jul 24 14:02:46.569427 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.569369 2579 scope.go:117] "RemoveContainer" containerID="bb80d92136a073f99c9df25d0d8e5e529912e8c3d7a675e2f60e397bdef029f6" Jul 24 14:02:46.569582 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:02:46.569567 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb80d92136a073f99c9df25d0d8e5e529912e8c3d7a675e2f60e397bdef029f6\": container with ID starting with bb80d92136a073f99c9df25d0d8e5e529912e8c3d7a675e2f60e397bdef029f6 not found: ID does not exist" containerID="bb80d92136a073f99c9df25d0d8e5e529912e8c3d7a675e2f60e397bdef029f6" Jul 24 14:02:46.569634 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.569587 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb80d92136a073f99c9df25d0d8e5e529912e8c3d7a675e2f60e397bdef029f6"} err="failed to get container status \"bb80d92136a073f99c9df25d0d8e5e529912e8c3d7a675e2f60e397bdef029f6\": rpc error: code = NotFound desc = could not find container \"bb80d92136a073f99c9df25d0d8e5e529912e8c3d7a675e2f60e397bdef029f6\": container with ID starting with bb80d92136a073f99c9df25d0d8e5e529912e8c3d7a675e2f60e397bdef029f6 not found: ID does not exist" Jul 24 14:02:46.619924 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.619887 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht"] Jul 24 14:02:46.628532 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.628503 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-4a9d360c-gw-openshift-default-6fddc76f88-5tkht"] Jul 24 14:02:46.635102 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.635066 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-podinfo\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:02:46.635102 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.635091 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-data\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:02:46.635102 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.635102 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-token\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:02:46.635382 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.635113 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kldxr\" (UniqueName: \"kubernetes.io/projected/74b70144-14b0-4bdd-9952-6fae60c0ca34-kube-api-access-kldxr\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:02:46.635382 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.635126 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74b70144-14b0-4bdd-9952-6fae60c0ca34-istio-envoy\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:02:46.640219 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:46.640188 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:48.420192 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:48.420157 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74b70144-14b0-4bdd-9952-6fae60c0ca34" path="/var/lib/kubelet/pods/74b70144-14b0-4bdd-9952-6fae60c0ca34/volumes" Jul 24 14:02:51.899302 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:51.899271 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:02:59.534059 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:59.533986 2579 patch_prober.go:28] interesting pod/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 24 14:02:59.534479 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:59.534094 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" podUID="835e3042-b9ad-4ff9-9462-4632d626b702" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 24 14:02:59.623702 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:59.623614 2579 generic.go:356] "Generic (PLEG): container finished" podID="835e3042-b9ad-4ff9-9462-4632d626b702" containerID="7566d1a6b3cdaad1616a834b6c8bbb12c79346f12e08be57efe3b539644af2d5" exitCode=0 Jul 24 14:02:59.623849 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:59.623690 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" event={"ID":"835e3042-b9ad-4ff9-9462-4632d626b702","Type":"ContainerDied","Data":"7566d1a6b3cdaad1616a834b6c8bbb12c79346f12e08be57efe3b539644af2d5"} Jul 24 14:02:59.624148 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:59.624119 2579 scope.go:117] "RemoveContainer" containerID="7566d1a6b3cdaad1616a834b6c8bbb12c79346f12e08be57efe3b539644af2d5" Jul 24 14:02:59.625235 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:59.625203 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:02:59.625348 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:59.625271 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:02:59.625348 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:02:59.625307 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:03:00.631462 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:00.631423 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" event={"ID":"835e3042-b9ad-4ff9-9462-4632d626b702","Type":"ContainerStarted","Data":"b96c848823662f42a46fd999074fab936750b2ffde43c03bfdf4864a829df952"} Jul 24 14:03:01.334475 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:01.334442 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:03:01.334678 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:01.334657 2579 patch_prober.go:28] interesting pod/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 24 14:03:01.334732 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:01.334714 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" podUID="835e3042-b9ad-4ff9-9462-4632d626b702" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 24 14:03:02.333898 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:02.333861 2579 patch_prober.go:28] interesting pod/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 24 14:03:02.334312 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:02.333919 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" podUID="835e3042-b9ad-4ff9-9462-4632d626b702" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 24 14:03:03.333837 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:03.333801 2579 patch_prober.go:28] interesting pod/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 24 14:03:03.334089 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:03.333854 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" podUID="835e3042-b9ad-4ff9-9462-4632d626b702" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 24 14:03:04.334580 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:04.334542 2579 patch_prober.go:28] interesting pod/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 24 14:03:04.335074 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:04.334601 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" podUID="835e3042-b9ad-4ff9-9462-4632d626b702" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 24 14:03:05.338172 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:05.338139 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:03:05.338551 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:05.338527 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:03:05.339281 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:05.339263 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:03:21.372600 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:21.372559 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:21.473491 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:21.473459 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:21.564939 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:21.564905 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:24.530510 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:24.530474 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:03:25.920984 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:25.920948 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:27.230299 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:27.230254 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg"] Jul 24 14:03:27.230722 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:27.230487 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" podUID="835e3042-b9ad-4ff9-9462-4632d626b702" containerName="istio-proxy" containerID="cri-o://b96c848823662f42a46fd999074fab936750b2ffde43c03bfdf4864a829df952" gracePeriod=30 Jul 24 14:03:27.260897 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:27.260863 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:27.571240 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:27.571197 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:27.755572 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:27.755499 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:28.086294 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.086259 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd"] Jul 24 14:03:28.086670 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.086658 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74b70144-14b0-4bdd-9952-6fae60c0ca34" containerName="istio-proxy" Jul 24 14:03:28.086714 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.086672 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="74b70144-14b0-4bdd-9952-6fae60c0ca34" containerName="istio-proxy" Jul 24 14:03:28.086714 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.086694 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad4f4258-65e7-44b1-88e5-3da42b8ead96" containerName="istio-proxy" Jul 24 14:03:28.086714 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.086699 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad4f4258-65e7-44b1-88e5-3da42b8ead96" containerName="istio-proxy" Jul 24 14:03:28.086857 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.086752 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad4f4258-65e7-44b1-88e5-3da42b8ead96" containerName="istio-proxy" Jul 24 14:03:28.086857 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.086759 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="74b70144-14b0-4bdd-9952-6fae60c0ca34" containerName="istio-proxy" Jul 24 14:03:28.089356 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.089339 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.093429 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.093413 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9846c499-openshift-default-dockercfg-d278w\"" Jul 24 14:03:28.121082 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.121055 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd"] Jul 24 14:03:28.125781 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.125758 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-credential-socket\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.125858 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.125807 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-workload-certs\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.125858 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.125837 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-envoy\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.125932 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.125858 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc6sp\" (UniqueName: \"kubernetes.io/projected/1b3962e6-a440-4994-80e4-f7f1843c0d5c-kube-api-access-cc6sp\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.125932 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.125885 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-podinfo\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.125932 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.125917 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istiod-ca-cert\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.126055 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.125961 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-token\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.126055 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.126026 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-workload-socket\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.126055 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.126046 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-data\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.167624 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.167573 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:28.226542 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.226504 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-podinfo\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.226721 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.226553 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istiod-ca-cert\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.226721 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.226597 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-token\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.226721 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.226630 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-workload-socket\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.226721 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.226659 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-data\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.226721 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.226693 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-credential-socket\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.227024 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.226739 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-workload-certs\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.227024 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.226759 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-envoy\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.227024 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.226775 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc6sp\" (UniqueName: \"kubernetes.io/projected/1b3962e6-a440-4994-80e4-f7f1843c0d5c-kube-api-access-cc6sp\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.227202 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.227108 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-workload-socket\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.227202 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.227129 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-data\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.227202 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.227184 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-workload-certs\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.227359 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.227254 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-credential-socket\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.227495 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.227479 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istiod-ca-cert\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.229333 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.229315 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-envoy\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.229618 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.229602 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-podinfo\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.240472 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.240448 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc6sp\" (UniqueName: \"kubernetes.io/projected/1b3962e6-a440-4994-80e4-f7f1843c0d5c-kube-api-access-cc6sp\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.240773 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.240500 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-token\") pod \"e2e-derive-9846c499-openshift-default-64f99b787b-82lvd\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.268060 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.268030 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:28.399539 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.399443 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:28.552661 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.552625 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd"] Jul 24 14:03:28.554108 ip-10-0-137-60 kubenswrapper[2579]: W0724 14:03:28.554081 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b3962e6_a440_4994_80e4_f7f1843c0d5c.slice/crio-921155bcce31f8354484657f8fc407f01fe511f3eb6f3cedb9c1062564998c96 WatchSource:0}: Error finding container 921155bcce31f8354484657f8fc407f01fe511f3eb6f3cedb9c1062564998c96: Status 404 returned error can't find the container with id 921155bcce31f8354484657f8fc407f01fe511f3eb6f3cedb9c1062564998c96 Jul 24 14:03:28.556243 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.556213 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:03:28.556292 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.556279 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:03:28.556331 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.556307 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:03:28.790852 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.790753 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" event={"ID":"1b3962e6-a440-4994-80e4-f7f1843c0d5c","Type":"ContainerStarted","Data":"8af1a1762f01aa9ed9a38b3a16400412c02f0284a131e92b8406113dc2332e1a"} Jul 24 14:03:28.790852 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:28.790816 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" event={"ID":"1b3962e6-a440-4994-80e4-f7f1843c0d5c","Type":"ContainerStarted","Data":"921155bcce31f8354484657f8fc407f01fe511f3eb6f3cedb9c1062564998c96"} Jul 24 14:03:29.399641 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:29.399605 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:29.404570 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:29.404542 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:29.436437 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:29.436388 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" podStartSLOduration=1.436371577 podStartE2EDuration="1.436371577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:03:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:03:28.831978122 +0000 UTC m=+1750.955360521" watchObservedRunningTime="2026-07-24 14:03:29.436371577 +0000 UTC m=+1751.559753975" Jul 24 14:03:29.802657 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:29.797028 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:29.803362 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:29.803334 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:30.336452 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:30.336415 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:30.468929 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:30.468893 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:30.641330 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:30.641239 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:30.858175 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:30.858138 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:31.299215 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:31.299176 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:31.505648 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:31.505612 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:32.496675 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.496610 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:03:32.566724 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.566693 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-workload-certs\") pod \"835e3042-b9ad-4ff9-9462-4632d626b702\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " Jul 24 14:03:32.567147 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.566756 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-credential-socket\") pod \"835e3042-b9ad-4ff9-9462-4632d626b702\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " Jul 24 14:03:32.567147 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.566801 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jl87h\" (UniqueName: \"kubernetes.io/projected/835e3042-b9ad-4ff9-9462-4632d626b702-kube-api-access-jl87h\") pod \"835e3042-b9ad-4ff9-9462-4632d626b702\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " Jul 24 14:03:32.567147 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.566819 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-workload-socket\") pod \"835e3042-b9ad-4ff9-9462-4632d626b702\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " Jul 24 14:03:32.567147 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.566836 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-istio-envoy\") pod \"835e3042-b9ad-4ff9-9462-4632d626b702\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " Jul 24 14:03:32.567147 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.566868 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-istio-data\") pod \"835e3042-b9ad-4ff9-9462-4632d626b702\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " Jul 24 14:03:32.567147 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.566930 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/835e3042-b9ad-4ff9-9462-4632d626b702-istio-token\") pod \"835e3042-b9ad-4ff9-9462-4632d626b702\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " Jul 24 14:03:32.567147 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.567035 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "835e3042-b9ad-4ff9-9462-4632d626b702" (UID: "835e3042-b9ad-4ff9-9462-4632d626b702"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:03:32.567724 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.567155 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "835e3042-b9ad-4ff9-9462-4632d626b702" (UID: "835e3042-b9ad-4ff9-9462-4632d626b702"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:03:32.567724 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.567174 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "835e3042-b9ad-4ff9-9462-4632d626b702" (UID: "835e3042-b9ad-4ff9-9462-4632d626b702"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:03:32.567724 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.566968 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/835e3042-b9ad-4ff9-9462-4632d626b702-istiod-ca-cert\") pod \"835e3042-b9ad-4ff9-9462-4632d626b702\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " Jul 24 14:03:32.567724 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.567304 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/835e3042-b9ad-4ff9-9462-4632d626b702-istio-podinfo\") pod \"835e3042-b9ad-4ff9-9462-4632d626b702\" (UID: \"835e3042-b9ad-4ff9-9462-4632d626b702\") " Jul 24 14:03:32.567724 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.567486 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/835e3042-b9ad-4ff9-9462-4632d626b702-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "835e3042-b9ad-4ff9-9462-4632d626b702" (UID: "835e3042-b9ad-4ff9-9462-4632d626b702"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:03:32.568118 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.567773 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-istio-data" (OuterVolumeSpecName: "istio-data") pod "835e3042-b9ad-4ff9-9462-4632d626b702" (UID: "835e3042-b9ad-4ff9-9462-4632d626b702"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:03:32.568118 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.567844 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-credential-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:03:32.568118 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.567863 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-workload-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:03:32.568118 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.567876 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/835e3042-b9ad-4ff9-9462-4632d626b702-istiod-ca-cert\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:03:32.568118 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.567890 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-workload-certs\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:03:32.569703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.569679 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/835e3042-b9ad-4ff9-9462-4632d626b702-istio-token" (OuterVolumeSpecName: "istio-token") pod "835e3042-b9ad-4ff9-9462-4632d626b702" (UID: "835e3042-b9ad-4ff9-9462-4632d626b702"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:03:32.569892 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.569873 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/835e3042-b9ad-4ff9-9462-4632d626b702-kube-api-access-jl87h" (OuterVolumeSpecName: "kube-api-access-jl87h") pod "835e3042-b9ad-4ff9-9462-4632d626b702" (UID: "835e3042-b9ad-4ff9-9462-4632d626b702"). InnerVolumeSpecName "kube-api-access-jl87h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:03:32.569965 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.569951 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/835e3042-b9ad-4ff9-9462-4632d626b702-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "835e3042-b9ad-4ff9-9462-4632d626b702" (UID: "835e3042-b9ad-4ff9-9462-4632d626b702"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:03:32.570116 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.570102 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "835e3042-b9ad-4ff9-9462-4632d626b702" (UID: "835e3042-b9ad-4ff9-9462-4632d626b702"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:03:32.668832 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.668795 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/835e3042-b9ad-4ff9-9462-4632d626b702-istio-token\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:03:32.668832 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.668828 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/835e3042-b9ad-4ff9-9462-4632d626b702-istio-podinfo\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:03:32.668832 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.668837 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jl87h\" (UniqueName: \"kubernetes.io/projected/835e3042-b9ad-4ff9-9462-4632d626b702-kube-api-access-jl87h\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:03:32.669107 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.668846 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-istio-envoy\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:03:32.669107 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.668856 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/835e3042-b9ad-4ff9-9462-4632d626b702-istio-data\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:03:32.815572 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.815540 2579 generic.go:356] "Generic (PLEG): container finished" podID="835e3042-b9ad-4ff9-9462-4632d626b702" containerID="b96c848823662f42a46fd999074fab936750b2ffde43c03bfdf4864a829df952" exitCode=0 Jul 24 14:03:32.815746 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.815592 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" Jul 24 14:03:32.815746 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.815622 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" event={"ID":"835e3042-b9ad-4ff9-9462-4632d626b702","Type":"ContainerDied","Data":"b96c848823662f42a46fd999074fab936750b2ffde43c03bfdf4864a829df952"} Jul 24 14:03:32.815746 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.815684 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg" event={"ID":"835e3042-b9ad-4ff9-9462-4632d626b702","Type":"ContainerDied","Data":"9c5bb88fca6ae47a93631310b2d8f78f0583b16965c7d09bc6c745ce77db60ec"} Jul 24 14:03:32.815746 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.815700 2579 scope.go:117] "RemoveContainer" containerID="b96c848823662f42a46fd999074fab936750b2ffde43c03bfdf4864a829df952" Jul 24 14:03:32.825746 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.825727 2579 scope.go:117] "RemoveContainer" containerID="7566d1a6b3cdaad1616a834b6c8bbb12c79346f12e08be57efe3b539644af2d5" Jul 24 14:03:32.834106 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.834087 2579 scope.go:117] "RemoveContainer" containerID="b96c848823662f42a46fd999074fab936750b2ffde43c03bfdf4864a829df952" Jul 24 14:03:32.834437 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:03:32.834418 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b96c848823662f42a46fd999074fab936750b2ffde43c03bfdf4864a829df952\": container with ID starting with b96c848823662f42a46fd999074fab936750b2ffde43c03bfdf4864a829df952 not found: ID does not exist" containerID="b96c848823662f42a46fd999074fab936750b2ffde43c03bfdf4864a829df952" Jul 24 14:03:32.834513 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.834446 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b96c848823662f42a46fd999074fab936750b2ffde43c03bfdf4864a829df952"} err="failed to get container status \"b96c848823662f42a46fd999074fab936750b2ffde43c03bfdf4864a829df952\": rpc error: code = NotFound desc = could not find container \"b96c848823662f42a46fd999074fab936750b2ffde43c03bfdf4864a829df952\": container with ID starting with b96c848823662f42a46fd999074fab936750b2ffde43c03bfdf4864a829df952 not found: ID does not exist" Jul 24 14:03:32.834513 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.834464 2579 scope.go:117] "RemoveContainer" containerID="7566d1a6b3cdaad1616a834b6c8bbb12c79346f12e08be57efe3b539644af2d5" Jul 24 14:03:32.834701 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:03:32.834686 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7566d1a6b3cdaad1616a834b6c8bbb12c79346f12e08be57efe3b539644af2d5\": container with ID starting with 7566d1a6b3cdaad1616a834b6c8bbb12c79346f12e08be57efe3b539644af2d5 not found: ID does not exist" containerID="7566d1a6b3cdaad1616a834b6c8bbb12c79346f12e08be57efe3b539644af2d5" Jul 24 14:03:32.834744 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.834706 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7566d1a6b3cdaad1616a834b6c8bbb12c79346f12e08be57efe3b539644af2d5"} err="failed to get container status \"7566d1a6b3cdaad1616a834b6c8bbb12c79346f12e08be57efe3b539644af2d5\": rpc error: code = NotFound desc = could not find container \"7566d1a6b3cdaad1616a834b6c8bbb12c79346f12e08be57efe3b539644af2d5\": container with ID starting with 7566d1a6b3cdaad1616a834b6c8bbb12c79346f12e08be57efe3b539644af2d5 not found: ID does not exist" Jul 24 14:03:32.851242 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.851211 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg"] Jul 24 14:03:32.853595 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:32.853569 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6538ff29-openshift-default-7d56d9cc4b-9dpkg"] Jul 24 14:03:34.419982 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:34.419945 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="835e3042-b9ad-4ff9-9462-4632d626b702" path="/var/lib/kubelet/pods/835e3042-b9ad-4ff9-9462-4632d626b702/volumes" Jul 24 14:03:35.926740 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:35.926708 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:03:43.891355 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:43.891320 2579 generic.go:356] "Generic (PLEG): container finished" podID="1b3962e6-a440-4994-80e4-f7f1843c0d5c" containerID="8af1a1762f01aa9ed9a38b3a16400412c02f0284a131e92b8406113dc2332e1a" exitCode=0 Jul 24 14:03:43.891795 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:43.891371 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" event={"ID":"1b3962e6-a440-4994-80e4-f7f1843c0d5c","Type":"ContainerDied","Data":"8af1a1762f01aa9ed9a38b3a16400412c02f0284a131e92b8406113dc2332e1a"} Jul 24 14:03:43.891795 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:43.891749 2579 scope.go:117] "RemoveContainer" containerID="8af1a1762f01aa9ed9a38b3a16400412c02f0284a131e92b8406113dc2332e1a" Jul 24 14:03:43.892767 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:43.892733 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:03:43.892879 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:43.892816 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:03:43.892879 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:43.892847 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:03:44.796575 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:44.796536 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:44.902312 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:44.902274 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" event={"ID":"1b3962e6-a440-4994-80e4-f7f1843c0d5c","Type":"ContainerStarted","Data":"50ae0fe21e56778616b02c9d35565f8ebf0cf1c6387a12d06e5e576dca3e8a4c"} Jul 24 14:03:45.399903 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:45.399873 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:45.401148 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:45.401120 2579 patch_prober.go:28] interesting pod/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 24 14:03:45.401293 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:45.401178 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" podUID="1b3962e6-a440-4994-80e4-f7f1843c0d5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 24 14:03:46.400223 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:46.400186 2579 patch_prober.go:28] interesting pod/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 24 14:03:46.400620 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:46.400260 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" podUID="1b3962e6-a440-4994-80e4-f7f1843c0d5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 24 14:03:47.400721 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:47.400686 2579 patch_prober.go:28] interesting pod/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 24 14:03:47.401175 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:47.400740 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" podUID="1b3962e6-a440-4994-80e4-f7f1843c0d5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 24 14:03:48.400478 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:48.400440 2579 patch_prober.go:28] interesting pod/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 24 14:03:48.400653 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:48.400504 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" podUID="1b3962e6-a440-4994-80e4-f7f1843c0d5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 24 14:03:49.403898 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:49.403870 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:49.404362 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:49.404255 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:03:49.405061 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:03:49.405042 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:04:03.010072 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:03.010035 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:04:03.109797 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:03.109756 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:04:06.306769 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:06.306732 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:04:06.446243 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:06.446203 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:04:06.549152 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:06.549104 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:04:09.426484 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:09.426398 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:04:10.341462 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:10.341424 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:04:10.500703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:10.500672 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd"] Jul 24 14:04:10.501113 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:10.500899 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" podUID="1b3962e6-a440-4994-80e4-f7f1843c0d5c" containerName="istio-proxy" containerID="cri-o://50ae0fe21e56778616b02c9d35565f8ebf0cf1c6387a12d06e5e576dca3e8a4c" gracePeriod=30 Jul 24 14:04:10.568510 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:10.568480 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:04:10.826305 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:10.826250 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:04:11.373906 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:11.373874 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:04:11.602954 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:11.602909 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:04:13.746680 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:13.746644 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:04:15.761524 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.761495 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:04:15.785465 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.785398 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-data\") pod \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " Jul 24 14:04:15.785465 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.785432 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-workload-certs\") pod \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " Jul 24 14:04:15.785465 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.785451 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-podinfo\") pod \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " Jul 24 14:04:15.785735 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.785486 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-envoy\") pod \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " Jul 24 14:04:15.785735 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.785618 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-credential-socket\") pod \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " Jul 24 14:04:15.785735 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.785692 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc6sp\" (UniqueName: \"kubernetes.io/projected/1b3962e6-a440-4994-80e4-f7f1843c0d5c-kube-api-access-cc6sp\") pod \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " Jul 24 14:04:15.785917 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.785744 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istiod-ca-cert\") pod \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " Jul 24 14:04:15.785917 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.785768 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-token\") pod \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " Jul 24 14:04:15.785917 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.785809 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1b3962e6-a440-4994-80e4-f7f1843c0d5c" (UID: "1b3962e6-a440-4994-80e4-f7f1843c0d5c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:04:15.785917 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.785832 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-workload-socket\") pod \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\" (UID: \"1b3962e6-a440-4994-80e4-f7f1843c0d5c\") " Jul 24 14:04:15.786535 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.786252 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-workload-certs\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:04:15.786535 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.786266 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-data" (OuterVolumeSpecName: "istio-data") pod "1b3962e6-a440-4994-80e4-f7f1843c0d5c" (UID: "1b3962e6-a440-4994-80e4-f7f1843c0d5c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:04:15.786535 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.786434 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1b3962e6-a440-4994-80e4-f7f1843c0d5c" (UID: "1b3962e6-a440-4994-80e4-f7f1843c0d5c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:04:15.786535 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.786489 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1b3962e6-a440-4994-80e4-f7f1843c0d5c" (UID: "1b3962e6-a440-4994-80e4-f7f1843c0d5c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:04:15.787027 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.786683 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1b3962e6-a440-4994-80e4-f7f1843c0d5c" (UID: "1b3962e6-a440-4994-80e4-f7f1843c0d5c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:04:15.788629 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.788576 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b3962e6-a440-4994-80e4-f7f1843c0d5c-kube-api-access-cc6sp" (OuterVolumeSpecName: "kube-api-access-cc6sp") pod "1b3962e6-a440-4994-80e4-f7f1843c0d5c" (UID: "1b3962e6-a440-4994-80e4-f7f1843c0d5c"). InnerVolumeSpecName "kube-api-access-cc6sp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:04:15.788754 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.788662 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-token" (OuterVolumeSpecName: "istio-token") pod "1b3962e6-a440-4994-80e4-f7f1843c0d5c" (UID: "1b3962e6-a440-4994-80e4-f7f1843c0d5c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:04:15.788891 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.788768 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1b3962e6-a440-4994-80e4-f7f1843c0d5c" (UID: "1b3962e6-a440-4994-80e4-f7f1843c0d5c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:04:15.789110 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.789093 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1b3962e6-a440-4994-80e4-f7f1843c0d5c" (UID: "1b3962e6-a440-4994-80e4-f7f1843c0d5c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:04:15.886863 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.886831 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istiod-ca-cert\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:04:15.886863 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.886856 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-token\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:04:15.886863 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.886865 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-workload-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:04:15.886863 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.886873 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-data\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:04:15.887197 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.886881 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-podinfo\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:04:15.887197 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.886888 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-istio-envoy\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:04:15.887197 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.886895 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3962e6-a440-4994-80e4-f7f1843c0d5c-credential-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:04:15.887197 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:15.886904 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc6sp\" (UniqueName: \"kubernetes.io/projected/1b3962e6-a440-4994-80e4-f7f1843c0d5c-kube-api-access-cc6sp\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:04:16.088265 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:16.088225 2579 generic.go:356] "Generic (PLEG): container finished" podID="1b3962e6-a440-4994-80e4-f7f1843c0d5c" containerID="50ae0fe21e56778616b02c9d35565f8ebf0cf1c6387a12d06e5e576dca3e8a4c" exitCode=0 Jul 24 14:04:16.088438 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:16.088288 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" Jul 24 14:04:16.088438 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:16.088313 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" event={"ID":"1b3962e6-a440-4994-80e4-f7f1843c0d5c","Type":"ContainerDied","Data":"50ae0fe21e56778616b02c9d35565f8ebf0cf1c6387a12d06e5e576dca3e8a4c"} Jul 24 14:04:16.088438 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:16.088375 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd" event={"ID":"1b3962e6-a440-4994-80e4-f7f1843c0d5c","Type":"ContainerDied","Data":"921155bcce31f8354484657f8fc407f01fe511f3eb6f3cedb9c1062564998c96"} Jul 24 14:04:16.088438 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:16.088398 2579 scope.go:117] "RemoveContainer" containerID="50ae0fe21e56778616b02c9d35565f8ebf0cf1c6387a12d06e5e576dca3e8a4c" Jul 24 14:04:16.098349 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:16.098332 2579 scope.go:117] "RemoveContainer" containerID="8af1a1762f01aa9ed9a38b3a16400412c02f0284a131e92b8406113dc2332e1a" Jul 24 14:04:16.109097 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:16.109071 2579 scope.go:117] "RemoveContainer" containerID="50ae0fe21e56778616b02c9d35565f8ebf0cf1c6387a12d06e5e576dca3e8a4c" Jul 24 14:04:16.109405 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:04:16.109380 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50ae0fe21e56778616b02c9d35565f8ebf0cf1c6387a12d06e5e576dca3e8a4c\": container with ID starting with 50ae0fe21e56778616b02c9d35565f8ebf0cf1c6387a12d06e5e576dca3e8a4c not found: ID does not exist" containerID="50ae0fe21e56778616b02c9d35565f8ebf0cf1c6387a12d06e5e576dca3e8a4c" Jul 24 14:04:16.109496 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:16.109418 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50ae0fe21e56778616b02c9d35565f8ebf0cf1c6387a12d06e5e576dca3e8a4c"} err="failed to get container status \"50ae0fe21e56778616b02c9d35565f8ebf0cf1c6387a12d06e5e576dca3e8a4c\": rpc error: code = NotFound desc = could not find container \"50ae0fe21e56778616b02c9d35565f8ebf0cf1c6387a12d06e5e576dca3e8a4c\": container with ID starting with 50ae0fe21e56778616b02c9d35565f8ebf0cf1c6387a12d06e5e576dca3e8a4c not found: ID does not exist" Jul 24 14:04:16.109496 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:16.109444 2579 scope.go:117] "RemoveContainer" containerID="8af1a1762f01aa9ed9a38b3a16400412c02f0284a131e92b8406113dc2332e1a" Jul 24 14:04:16.109724 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:04:16.109706 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8af1a1762f01aa9ed9a38b3a16400412c02f0284a131e92b8406113dc2332e1a\": container with ID starting with 8af1a1762f01aa9ed9a38b3a16400412c02f0284a131e92b8406113dc2332e1a not found: ID does not exist" containerID="8af1a1762f01aa9ed9a38b3a16400412c02f0284a131e92b8406113dc2332e1a" Jul 24 14:04:16.109765 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:16.109731 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8af1a1762f01aa9ed9a38b3a16400412c02f0284a131e92b8406113dc2332e1a"} err="failed to get container status \"8af1a1762f01aa9ed9a38b3a16400412c02f0284a131e92b8406113dc2332e1a\": rpc error: code = NotFound desc = could not find container \"8af1a1762f01aa9ed9a38b3a16400412c02f0284a131e92b8406113dc2332e1a\": container with ID starting with 8af1a1762f01aa9ed9a38b3a16400412c02f0284a131e92b8406113dc2332e1a not found: ID does not exist" Jul 24 14:04:16.116406 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:16.116379 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd"] Jul 24 14:04:16.122515 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:16.122488 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9846c499-openshift-default-64f99b787b-82lvd"] Jul 24 14:04:16.420566 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:16.420477 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b3962e6-a440-4994-80e4-f7f1843c0d5c" path="/var/lib/kubelet/pods/1b3962e6-a440-4994-80e4-f7f1843c0d5c/volumes" Jul 24 14:04:20.355526 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:20.355479 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:04:20.475782 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:20.475747 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:04:20.581182 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:20.581141 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:04:20.683211 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:20.683122 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:04:26.268948 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:26.268912 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:04:52.282752 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:04:52.282714 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:05:01.365371 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:05:01.365303 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:05:01.553844 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:05:01.553812 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:05:01.750632 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:05:01.750553 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:05:17.646059 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:05:17.646028 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:05:22.772865 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:05:22.772829 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:05:24.436159 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:05:24.436129 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:05:24.592959 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:05:24.592926 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:05:24.778605 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:05:24.778299 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:05:35.281409 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:05:35.281368 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:05:36.164462 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:05:36.164433 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:05:43.888813 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:05:43.888716 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:05:45.282448 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:05:45.282414 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:05:47.143052 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:05:47.142969 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:05:47.850335 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:05:47.850294 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:05:54.232187 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:05:54.232145 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:05:54.342716 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:05:54.342681 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:05:54.984001 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:05:54.983969 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:05:58.014965 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:05:58.014930 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:06:02.021080 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.021042 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:02.134811 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.134776 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:02.282175 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.282082 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:02.616571 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.616538 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6"] Jul 24 14:06:02.616938 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.616923 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b3962e6-a440-4994-80e4-f7f1843c0d5c" containerName="istio-proxy" Jul 24 14:06:02.616986 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.616940 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b3962e6-a440-4994-80e4-f7f1843c0d5c" containerName="istio-proxy" Jul 24 14:06:02.616986 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.616951 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="835e3042-b9ad-4ff9-9462-4632d626b702" containerName="istio-proxy" Jul 24 14:06:02.616986 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.616956 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="835e3042-b9ad-4ff9-9462-4632d626b702" containerName="istio-proxy" Jul 24 14:06:02.616986 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.616964 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b3962e6-a440-4994-80e4-f7f1843c0d5c" containerName="istio-proxy" Jul 24 14:06:02.616986 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.616970 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b3962e6-a440-4994-80e4-f7f1843c0d5c" containerName="istio-proxy" Jul 24 14:06:02.617205 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.617003 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74b70144-14b0-4bdd-9952-6fae60c0ca34" containerName="istio-proxy" Jul 24 14:06:02.617205 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.617011 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="74b70144-14b0-4bdd-9952-6fae60c0ca34" containerName="istio-proxy" Jul 24 14:06:02.617205 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.617107 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b3962e6-a440-4994-80e4-f7f1843c0d5c" containerName="istio-proxy" Jul 24 14:06:02.617205 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.617122 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="835e3042-b9ad-4ff9-9462-4632d626b702" containerName="istio-proxy" Jul 24 14:06:02.617205 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.617134 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="74b70144-14b0-4bdd-9952-6fae60c0ca34" containerName="istio-proxy" Jul 24 14:06:02.617205 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.617143 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b3962e6-a440-4994-80e4-f7f1843c0d5c" containerName="istio-proxy" Jul 24 14:06:02.617205 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.617155 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="835e3042-b9ad-4ff9-9462-4632d626b702" containerName="istio-proxy" Jul 24 14:06:02.620405 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.620383 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.624052 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.624035 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-296b2828-openshift-default-dockercfg-gxltn\"" Jul 24 14:06:02.647613 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.647578 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6"] Jul 24 14:06:02.648670 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.648644 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:02.703486 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.703448 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-token\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.703663 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.703494 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-podinfo\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.703663 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.703576 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-data\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.703663 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.703611 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2559e564-0b80-423a-8d72-9c9ac81032f6-istiod-ca-cert\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.703663 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.703642 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-workload-certs\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.703889 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.703681 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-workload-socket\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.703889 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.703754 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-envoy\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.703889 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.703788 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-credential-socket\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.703889 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.703814 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7v66\" (UniqueName: \"kubernetes.io/projected/2559e564-0b80-423a-8d72-9c9ac81032f6-kube-api-access-d7v66\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.805044 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.804978 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-envoy\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.805044 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.805043 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-credential-socket\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.805312 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.805064 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7v66\" (UniqueName: \"kubernetes.io/projected/2559e564-0b80-423a-8d72-9c9ac81032f6-kube-api-access-d7v66\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.805312 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.805115 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-token\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.805312 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.805138 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-podinfo\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.805312 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.805176 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-data\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.805312 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.805204 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2559e564-0b80-423a-8d72-9c9ac81032f6-istiod-ca-cert\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.805312 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.805238 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-workload-certs\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.805312 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.805271 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-workload-socket\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.805735 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.805526 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-credential-socket\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.805735 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.805605 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-data\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.805735 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.805632 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-workload-certs\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.805735 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.805676 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-workload-socket\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.806025 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.805978 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2559e564-0b80-423a-8d72-9c9ac81032f6-istiod-ca-cert\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.807643 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.807627 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-envoy\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.807853 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.807832 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-podinfo\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.816559 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.816532 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7v66\" (UniqueName: \"kubernetes.io/projected/2559e564-0b80-423a-8d72-9c9ac81032f6-kube-api-access-d7v66\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.816893 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.816877 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-token\") pod \"e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:02.931251 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:02.931144 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:03.081887 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:03.081862 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6"] Jul 24 14:06:03.084478 ip-10-0-137-60 kubenswrapper[2579]: W0724 14:06:03.084451 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2559e564_0b80_423a_8d72_9c9ac81032f6.slice/crio-240e2128971fc40b8848cf13a2faf16b6c129c4310d582034aefa636a9f7ff68 WatchSource:0}: Error finding container 240e2128971fc40b8848cf13a2faf16b6c129c4310d582034aefa636a9f7ff68: Status 404 returned error can't find the container with id 240e2128971fc40b8848cf13a2faf16b6c129c4310d582034aefa636a9f7ff68 Jul 24 14:06:03.086639 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:03.086600 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:06:03.086741 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:03.086668 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:06:03.086741 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:03.086694 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:06:03.173467 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:03.173431 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:03.666496 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:03.666460 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" event={"ID":"2559e564-0b80-423a-8d72-9c9ac81032f6","Type":"ContainerStarted","Data":"50d13c0faa4b7aebbede7191c5818cf4e1445f7e733bc86abd09f90c219d9919"} Jul 24 14:06:03.666659 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:03.666517 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" event={"ID":"2559e564-0b80-423a-8d72-9c9ac81032f6","Type":"ContainerStarted","Data":"240e2128971fc40b8848cf13a2faf16b6c129c4310d582034aefa636a9f7ff68"} Jul 24 14:06:03.698072 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:03.698027 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" podStartSLOduration=1.697984859 podStartE2EDuration="1.697984859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:06:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:06:03.694246965 +0000 UTC m=+1905.817629360" watchObservedRunningTime="2026-07-24 14:06:03.697984859 +0000 UTC m=+1905.821367320" Jul 24 14:06:03.931386 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:03.931294 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:04.937593 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:04.937563 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:05.684231 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:05.684198 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:05.685900 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:05.685875 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:05.751554 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:05.751377 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:05.857858 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:05.857823 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:05.943624 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:05.943524 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:06.594601 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:06.594566 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:06.790516 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:06.790481 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:06.884971 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:06.884875 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:10.266367 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:10.266327 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:11.015228 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:11.015183 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:11.110725 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:11.110661 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:11.225210 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:11.225171 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:06:11.483013 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:11.482952 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:17.943540 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:17.943497 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:18.030538 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:18.030502 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:19.759061 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:19.758945 2579 generic.go:356] "Generic (PLEG): container finished" podID="2559e564-0b80-423a-8d72-9c9ac81032f6" containerID="50d13c0faa4b7aebbede7191c5818cf4e1445f7e733bc86abd09f90c219d9919" exitCode=0 Jul 24 14:06:19.759061 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:19.759032 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" event={"ID":"2559e564-0b80-423a-8d72-9c9ac81032f6","Type":"ContainerDied","Data":"50d13c0faa4b7aebbede7191c5818cf4e1445f7e733bc86abd09f90c219d9919"} Jul 24 14:06:19.759595 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:19.759503 2579 scope.go:117] "RemoveContainer" containerID="50d13c0faa4b7aebbede7191c5818cf4e1445f7e733bc86abd09f90c219d9919" Jul 24 14:06:19.760950 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:19.760913 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:06:19.761090 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:19.761031 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:06:19.761090 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:19.761068 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:06:20.679937 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:20.679900 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:20.771062 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:20.771024 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" event={"ID":"2559e564-0b80-423a-8d72-9c9ac81032f6","Type":"ContainerStarted","Data":"f139d71220704ee78cf3159532154dae599191a5820422eef58b7fbad40efc52"} Jul 24 14:06:20.931880 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:20.931785 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:20.932985 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:20.932950 2579 patch_prober.go:28] interesting pod/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 24 14:06:20.933159 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:20.933024 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" podUID="2559e564-0b80-423a-8d72-9c9ac81032f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 24 14:06:21.931567 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:21.931521 2579 patch_prober.go:28] interesting pod/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 24 14:06:21.932079 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:21.931631 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" podUID="2559e564-0b80-423a-8d72-9c9ac81032f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 24 14:06:22.931898 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:22.931864 2579 patch_prober.go:28] interesting pod/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 24 14:06:22.932285 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:22.931918 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" podUID="2559e564-0b80-423a-8d72-9c9ac81032f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 24 14:06:23.932329 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:23.932284 2579 patch_prober.go:28] interesting pod/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 24 14:06:23.932751 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:23.932352 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" podUID="2559e564-0b80-423a-8d72-9c9ac81032f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 24 14:06:24.936245 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:24.936214 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:24.936716 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:24.936503 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:24.937335 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:24.937314 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:28.858753 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:28.858718 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:28.993407 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:28.993377 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:29.089954 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:29.089918 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:32.118543 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:32.118506 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:06:35.315182 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:35.315137 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:35.415718 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:35.415677 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:40.291054 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:40.291016 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:40.392174 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:40.392137 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:40.610647 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:40.610611 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6"] Jul 24 14:06:40.610854 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:40.610829 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" podUID="2559e564-0b80-423a-8d72-9c9ac81032f6" containerName="istio-proxy" containerID="cri-o://f139d71220704ee78cf3159532154dae599191a5820422eef58b7fbad40efc52" gracePeriod=30 Jul 24 14:06:40.661026 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:40.660971 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:40.912733 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:40.912646 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:41.033805 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:41.033767 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:41.137703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:41.137667 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:41.302967 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:41.302880 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:45.863646 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:45.863620 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:45.916799 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:45.916696 2579 generic.go:356] "Generic (PLEG): container finished" podID="2559e564-0b80-423a-8d72-9c9ac81032f6" containerID="f139d71220704ee78cf3159532154dae599191a5820422eef58b7fbad40efc52" exitCode=0 Jul 24 14:06:45.916799 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:45.916764 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" Jul 24 14:06:45.916799 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:45.916781 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" event={"ID":"2559e564-0b80-423a-8d72-9c9ac81032f6","Type":"ContainerDied","Data":"f139d71220704ee78cf3159532154dae599191a5820422eef58b7fbad40efc52"} Jul 24 14:06:45.917125 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:45.916844 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6" event={"ID":"2559e564-0b80-423a-8d72-9c9ac81032f6","Type":"ContainerDied","Data":"240e2128971fc40b8848cf13a2faf16b6c129c4310d582034aefa636a9f7ff68"} Jul 24 14:06:45.917125 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:45.916874 2579 scope.go:117] "RemoveContainer" containerID="f139d71220704ee78cf3159532154dae599191a5820422eef58b7fbad40efc52" Jul 24 14:06:45.928233 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:45.928207 2579 scope.go:117] "RemoveContainer" containerID="50d13c0faa4b7aebbede7191c5818cf4e1445f7e733bc86abd09f90c219d9919" Jul 24 14:06:45.937032 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:45.937006 2579 scope.go:117] "RemoveContainer" containerID="f139d71220704ee78cf3159532154dae599191a5820422eef58b7fbad40efc52" Jul 24 14:06:45.937290 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:06:45.937268 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f139d71220704ee78cf3159532154dae599191a5820422eef58b7fbad40efc52\": container with ID starting with f139d71220704ee78cf3159532154dae599191a5820422eef58b7fbad40efc52 not found: ID does not exist" containerID="f139d71220704ee78cf3159532154dae599191a5820422eef58b7fbad40efc52" Jul 24 14:06:45.937359 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:45.937299 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f139d71220704ee78cf3159532154dae599191a5820422eef58b7fbad40efc52"} err="failed to get container status \"f139d71220704ee78cf3159532154dae599191a5820422eef58b7fbad40efc52\": rpc error: code = NotFound desc = could not find container \"f139d71220704ee78cf3159532154dae599191a5820422eef58b7fbad40efc52\": container with ID starting with f139d71220704ee78cf3159532154dae599191a5820422eef58b7fbad40efc52 not found: ID does not exist" Jul 24 14:06:45.937359 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:45.937317 2579 scope.go:117] "RemoveContainer" containerID="50d13c0faa4b7aebbede7191c5818cf4e1445f7e733bc86abd09f90c219d9919" Jul 24 14:06:45.937555 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:06:45.937540 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50d13c0faa4b7aebbede7191c5818cf4e1445f7e733bc86abd09f90c219d9919\": container with ID starting with 50d13c0faa4b7aebbede7191c5818cf4e1445f7e733bc86abd09f90c219d9919 not found: ID does not exist" containerID="50d13c0faa4b7aebbede7191c5818cf4e1445f7e733bc86abd09f90c219d9919" Jul 24 14:06:45.937599 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:45.937557 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50d13c0faa4b7aebbede7191c5818cf4e1445f7e733bc86abd09f90c219d9919"} err="failed to get container status \"50d13c0faa4b7aebbede7191c5818cf4e1445f7e733bc86abd09f90c219d9919\": rpc error: code = NotFound desc = could not find container \"50d13c0faa4b7aebbede7191c5818cf4e1445f7e733bc86abd09f90c219d9919\": container with ID starting with 50d13c0faa4b7aebbede7191c5818cf4e1445f7e733bc86abd09f90c219d9919 not found: ID does not exist" Jul 24 14:06:46.027306 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.027263 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-token\") pod \"2559e564-0b80-423a-8d72-9c9ac81032f6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " Jul 24 14:06:46.027306 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.027314 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-workload-certs\") pod \"2559e564-0b80-423a-8d72-9c9ac81032f6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " Jul 24 14:06:46.027553 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.027339 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-workload-socket\") pod \"2559e564-0b80-423a-8d72-9c9ac81032f6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " Jul 24 14:06:46.027553 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.027361 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-podinfo\") pod \"2559e564-0b80-423a-8d72-9c9ac81032f6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " Jul 24 14:06:46.027553 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.027415 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2559e564-0b80-423a-8d72-9c9ac81032f6-istiod-ca-cert\") pod \"2559e564-0b80-423a-8d72-9c9ac81032f6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " Jul 24 14:06:46.027553 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.027480 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-data\") pod \"2559e564-0b80-423a-8d72-9c9ac81032f6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " Jul 24 14:06:46.027553 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.027541 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-credential-socket\") pod \"2559e564-0b80-423a-8d72-9c9ac81032f6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " Jul 24 14:06:46.027797 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.027576 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7v66\" (UniqueName: \"kubernetes.io/projected/2559e564-0b80-423a-8d72-9c9ac81032f6-kube-api-access-d7v66\") pod \"2559e564-0b80-423a-8d72-9c9ac81032f6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " Jul 24 14:06:46.027797 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.027599 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-envoy\") pod \"2559e564-0b80-423a-8d72-9c9ac81032f6\" (UID: \"2559e564-0b80-423a-8d72-9c9ac81032f6\") " Jul 24 14:06:46.027797 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.027679 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2559e564-0b80-423a-8d72-9c9ac81032f6" (UID: "2559e564-0b80-423a-8d72-9c9ac81032f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:06:46.027941 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.027834 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2559e564-0b80-423a-8d72-9c9ac81032f6" (UID: "2559e564-0b80-423a-8d72-9c9ac81032f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:06:46.027941 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.027687 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2559e564-0b80-423a-8d72-9c9ac81032f6" (UID: "2559e564-0b80-423a-8d72-9c9ac81032f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:06:46.028053 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.027981 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-credential-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:06:46.028053 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.028037 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-workload-certs\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:06:46.028143 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.028057 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-workload-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:06:46.028522 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.028490 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2559e564-0b80-423a-8d72-9c9ac81032f6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2559e564-0b80-423a-8d72-9c9ac81032f6" (UID: "2559e564-0b80-423a-8d72-9c9ac81032f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:06:46.028773 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.028735 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-data" (OuterVolumeSpecName: "istio-data") pod "2559e564-0b80-423a-8d72-9c9ac81032f6" (UID: "2559e564-0b80-423a-8d72-9c9ac81032f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:06:46.029794 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.029765 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2559e564-0b80-423a-8d72-9c9ac81032f6" (UID: "2559e564-0b80-423a-8d72-9c9ac81032f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:06:46.029876 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.029814 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-token" (OuterVolumeSpecName: "istio-token") pod "2559e564-0b80-423a-8d72-9c9ac81032f6" (UID: "2559e564-0b80-423a-8d72-9c9ac81032f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:06:46.029932 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.029872 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2559e564-0b80-423a-8d72-9c9ac81032f6" (UID: "2559e564-0b80-423a-8d72-9c9ac81032f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:06:46.030211 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.030188 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2559e564-0b80-423a-8d72-9c9ac81032f6-kube-api-access-d7v66" (OuterVolumeSpecName: "kube-api-access-d7v66") pod "2559e564-0b80-423a-8d72-9c9ac81032f6" (UID: "2559e564-0b80-423a-8d72-9c9ac81032f6"). InnerVolumeSpecName "kube-api-access-d7v66". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:06:46.129450 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.129412 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-podinfo\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:06:46.129450 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.129444 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2559e564-0b80-423a-8d72-9c9ac81032f6-istiod-ca-cert\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:06:46.129450 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.129452 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-data\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:06:46.129450 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.129460 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7v66\" (UniqueName: \"kubernetes.io/projected/2559e564-0b80-423a-8d72-9c9ac81032f6-kube-api-access-d7v66\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:06:46.129716 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.129469 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-envoy\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:06:46.129716 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.129477 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2559e564-0b80-423a-8d72-9c9ac81032f6-istio-token\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:06:46.246372 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.246343 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6"] Jul 24 14:06:46.252805 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.252773 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-296b2828-openshift-default-758c6bc6b4-hzjt6"] Jul 24 14:06:46.419719 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:46.419682 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2559e564-0b80-423a-8d72-9c9ac81032f6" path="/var/lib/kubelet/pods/2559e564-0b80-423a-8d72-9c9ac81032f6/volumes" Jul 24 14:06:47.565639 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:47.565602 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:47.668451 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:47.668416 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:47.767056 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:47.767012 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:47.946183 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:47.946095 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:48.125092 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:48.125057 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:54.510176 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:54.510132 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:06:54.626440 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:06:54.626404 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:07:01.504765 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.504727 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:07:01.837115 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.837081 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59"] Jul 24 14:07:01.837470 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.837457 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2559e564-0b80-423a-8d72-9c9ac81032f6" containerName="istio-proxy" Jul 24 14:07:01.837523 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.837473 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2559e564-0b80-423a-8d72-9c9ac81032f6" containerName="istio-proxy" Jul 24 14:07:01.837523 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.837486 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2559e564-0b80-423a-8d72-9c9ac81032f6" containerName="istio-proxy" Jul 24 14:07:01.837523 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.837492 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2559e564-0b80-423a-8d72-9c9ac81032f6" containerName="istio-proxy" Jul 24 14:07:01.837523 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.837512 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="835e3042-b9ad-4ff9-9462-4632d626b702" containerName="istio-proxy" Jul 24 14:07:01.837523 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.837518 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="835e3042-b9ad-4ff9-9462-4632d626b702" containerName="istio-proxy" Jul 24 14:07:01.837682 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.837574 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2559e564-0b80-423a-8d72-9c9ac81032f6" containerName="istio-proxy" Jul 24 14:07:01.837682 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.837580 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2559e564-0b80-423a-8d72-9c9ac81032f6" containerName="istio-proxy" Jul 24 14:07:01.841251 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.841233 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:01.844689 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.844661 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3098bd50-openshift-default-dockercfg-jxb6l\"" Jul 24 14:07:01.853417 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.853389 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59"] Jul 24 14:07:01.865417 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.865388 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:07:01.949592 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.949549 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wbwp\" (UniqueName: \"kubernetes.io/projected/c519ff6c-5b5f-4383-ab09-fca419176a29-kube-api-access-8wbwp\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:01.949788 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.949602 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c519ff6c-5b5f-4383-ab09-fca419176a29-istiod-ca-cert\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:01.949788 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.949684 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-data\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:01.949788 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.949748 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-podinfo\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:01.949968 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.949790 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-workload-certs\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:01.949968 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.949839 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-workload-socket\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:01.949968 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.949917 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-envoy\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:01.950160 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.949978 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-credential-socket\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:01.950160 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:01.950019 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-token\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.050560 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.050525 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c519ff6c-5b5f-4383-ab09-fca419176a29-istiod-ca-cert\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.050756 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.050574 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-data\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.050756 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.050724 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-podinfo\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.050896 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.050788 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-workload-certs\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.050896 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.050830 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-workload-socket\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.051043 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.050926 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-envoy\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.051043 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.050974 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-data\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.051043 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.051019 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-credential-socket\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.051214 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.051047 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-token\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.051214 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.051095 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wbwp\" (UniqueName: \"kubernetes.io/projected/c519ff6c-5b5f-4383-ab09-fca419176a29-kube-api-access-8wbwp\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.051214 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.051100 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-workload-certs\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.051366 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.051230 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-workload-socket\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.051427 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.051383 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c519ff6c-5b5f-4383-ab09-fca419176a29-istiod-ca-cert\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.051427 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.051407 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-credential-socket\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.053551 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.053519 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-envoy\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.053677 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.053591 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-podinfo\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.059962 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.059937 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-token\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.059962 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.059951 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wbwp\" (UniqueName: \"kubernetes.io/projected/c519ff6c-5b5f-4383-ab09-fca419176a29-kube-api-access-8wbwp\") pod \"e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.154311 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.154219 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:02.296249 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.296216 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59"] Jul 24 14:07:02.297470 ip-10-0-137-60 kubenswrapper[2579]: W0724 14:07:02.297443 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc519ff6c_5b5f_4383_ab09_fca419176a29.slice/crio-d8fef1513b1b36fa73617a4f45e22437d50772c528e37731fec11689f42bed60 WatchSource:0}: Error finding container d8fef1513b1b36fa73617a4f45e22437d50772c528e37731fec11689f42bed60: Status 404 returned error can't find the container with id d8fef1513b1b36fa73617a4f45e22437d50772c528e37731fec11689f42bed60 Jul 24 14:07:02.299748 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.299712 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:07:02.299844 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.299825 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:07:02.299889 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.299874 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:07:02.427162 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:02.427075 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:07:03.010718 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:03.010678 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" event={"ID":"c519ff6c-5b5f-4383-ab09-fca419176a29","Type":"ContainerStarted","Data":"a6118b5be68d3e7959df5be108edb1b227e98db242f70e90e9dadcb66b822b04"} Jul 24 14:07:03.011153 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:03.010743 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" event={"ID":"c519ff6c-5b5f-4383-ab09-fca419176a29","Type":"ContainerStarted","Data":"d8fef1513b1b36fa73617a4f45e22437d50772c528e37731fec11689f42bed60"} Jul 24 14:07:03.036465 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:03.036412 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" podStartSLOduration=2.036397258 podStartE2EDuration="2.036397258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:07:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:07:03.034275011 +0000 UTC m=+1965.157657409" watchObservedRunningTime="2026-07-24 14:07:03.036397258 +0000 UTC m=+1965.159779654" Jul 24 14:07:03.154872 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:03.154836 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:03.160884 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:03.160852 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:04.016647 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:04.016607 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:04.017581 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:04.017561 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:05.081216 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:05.081167 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:07:05.210895 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:05.210860 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:07:05.326102 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:05.326055 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:07:05.483011 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:05.482915 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:07:06.171296 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:06.171245 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:07:06.266855 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:06.266819 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:07:09.577164 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:09.577077 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:07:09.619027 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:09.618980 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:07:18.111663 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:18.111624 2579 generic.go:356] "Generic (PLEG): container finished" podID="c519ff6c-5b5f-4383-ab09-fca419176a29" containerID="a6118b5be68d3e7959df5be108edb1b227e98db242f70e90e9dadcb66b822b04" exitCode=0 Jul 24 14:07:18.112146 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:18.111700 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" event={"ID":"c519ff6c-5b5f-4383-ab09-fca419176a29","Type":"ContainerDied","Data":"a6118b5be68d3e7959df5be108edb1b227e98db242f70e90e9dadcb66b822b04"} Jul 24 14:07:18.112146 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:18.112117 2579 scope.go:117] "RemoveContainer" containerID="a6118b5be68d3e7959df5be108edb1b227e98db242f70e90e9dadcb66b822b04" Jul 24 14:07:18.113120 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:18.113091 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:07:18.113197 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:18.113168 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:07:18.113234 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:18.113199 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:07:19.017514 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:19.017472 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:19.119504 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:19.119468 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" event={"ID":"c519ff6c-5b5f-4383-ab09-fca419176a29","Type":"ContainerStarted","Data":"d015bfea342ae779d2792e183f261e8760370dd22dd928cffb4527144e382055"} Jul 24 14:07:19.155215 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:19.155172 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:19.156440 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:19.156401 2579 patch_prober.go:28] interesting pod/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 24 14:07:19.156574 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:19.156487 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" podUID="c519ff6c-5b5f-4383-ab09-fca419176a29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 24 14:07:20.155419 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:20.155381 2579 patch_prober.go:28] interesting pod/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 24 14:07:20.156031 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:20.155979 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" podUID="c519ff6c-5b5f-4383-ab09-fca419176a29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 24 14:07:21.155774 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:21.155725 2579 patch_prober.go:28] interesting pod/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 24 14:07:21.156322 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:21.155838 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" podUID="c519ff6c-5b5f-4383-ab09-fca419176a29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 24 14:07:22.154938 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:22.154885 2579 patch_prober.go:28] interesting pod/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 24 14:07:22.155183 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:22.154961 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" podUID="c519ff6c-5b5f-4383-ab09-fca419176a29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 24 14:07:23.159377 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:23.159346 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:23.159757 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:23.159599 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:23.160254 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:23.160232 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:07:39.737077 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:39.737046 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:07:42.861617 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:07:42.861579 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:08:00.606419 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:08:00.606387 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:08:19.782329 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:08:19.782286 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:08:20.820894 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:08:20.820858 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:08:20.987247 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:08:20.987214 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:08:21.154609 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:08:21.154487 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:08:24.208914 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:08:24.208874 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:08:25.107288 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:08:25.107254 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:08:28.727275 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:08:28.727233 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:08:29.039581 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:08:29.039498 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:08:34.159573 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:08:34.159534 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:11.291821 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:11.291783 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:13.102248 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:13.102216 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:19.534697 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:19.534664 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:19.632190 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:19.632155 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:19.854819 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:19.854783 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59"] Jul 24 14:09:19.855092 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:19.855068 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" podUID="c519ff6c-5b5f-4383-ab09-fca419176a29" containerName="istio-proxy" containerID="cri-o://d015bfea342ae779d2792e183f261e8760370dd22dd928cffb4527144e382055" gracePeriod=30 Jul 24 14:09:19.909942 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:19.909899 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:20.180269 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:20.180186 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:21.295778 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:21.295743 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:21.395446 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:21.395409 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:21.496946 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:21.496912 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:22.469871 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:22.469830 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:22.676643 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:22.676605 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:23.162634 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:23.162598 2579 patch_prober.go:28] interesting pod/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 14:09:23.162802 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:23.162661 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" podUID="c519ff6c-5b5f-4383-ab09-fca419176a29" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 14:09:23.985510 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:23.985478 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:09:24.515937 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:24.515899 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:25.114075 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.113985 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:09:25.234097 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.234060 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-data\") pod \"c519ff6c-5b5f-4383-ab09-fca419176a29\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " Jul 24 14:09:25.234276 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.234122 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-workload-socket\") pod \"c519ff6c-5b5f-4383-ab09-fca419176a29\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " Jul 24 14:09:25.234276 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.234174 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wbwp\" (UniqueName: \"kubernetes.io/projected/c519ff6c-5b5f-4383-ab09-fca419176a29-kube-api-access-8wbwp\") pod \"c519ff6c-5b5f-4383-ab09-fca419176a29\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " Jul 24 14:09:25.234276 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.234254 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-envoy\") pod \"c519ff6c-5b5f-4383-ab09-fca419176a29\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " Jul 24 14:09:25.234530 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.234284 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c519ff6c-5b5f-4383-ab09-fca419176a29-istiod-ca-cert\") pod \"c519ff6c-5b5f-4383-ab09-fca419176a29\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " Jul 24 14:09:25.234530 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.234444 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-workload-certs\") pod \"c519ff6c-5b5f-4383-ab09-fca419176a29\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " Jul 24 14:09:25.234530 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.234485 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-credential-socket\") pod \"c519ff6c-5b5f-4383-ab09-fca419176a29\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " Jul 24 14:09:25.234738 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.234534 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-token\") pod \"c519ff6c-5b5f-4383-ab09-fca419176a29\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " Jul 24 14:09:25.234738 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.234541 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c519ff6c-5b5f-4383-ab09-fca419176a29" (UID: "c519ff6c-5b5f-4383-ab09-fca419176a29"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:09:25.234738 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.234577 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-podinfo\") pod \"c519ff6c-5b5f-4383-ab09-fca419176a29\" (UID: \"c519ff6c-5b5f-4383-ab09-fca419176a29\") " Jul 24 14:09:25.234895 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.234732 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c519ff6c-5b5f-4383-ab09-fca419176a29" (UID: "c519ff6c-5b5f-4383-ab09-fca419176a29"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:09:25.234895 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.234785 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c519ff6c-5b5f-4383-ab09-fca419176a29" (UID: "c519ff6c-5b5f-4383-ab09-fca419176a29"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:09:25.235025 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.234974 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-workload-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:09:25.235154 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.235023 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-workload-certs\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:09:25.235154 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.235039 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-credential-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:09:25.235154 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.235049 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c519ff6c-5b5f-4383-ab09-fca419176a29-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c519ff6c-5b5f-4383-ab09-fca419176a29" (UID: "c519ff6c-5b5f-4383-ab09-fca419176a29"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:09:25.235417 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.235300 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-data" (OuterVolumeSpecName: "istio-data") pod "c519ff6c-5b5f-4383-ab09-fca419176a29" (UID: "c519ff6c-5b5f-4383-ab09-fca419176a29"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:09:25.236730 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.236699 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c519ff6c-5b5f-4383-ab09-fca419176a29" (UID: "c519ff6c-5b5f-4383-ab09-fca419176a29"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:09:25.236852 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.236746 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c519ff6c-5b5f-4383-ab09-fca419176a29-kube-api-access-8wbwp" (OuterVolumeSpecName: "kube-api-access-8wbwp") pod "c519ff6c-5b5f-4383-ab09-fca419176a29" (UID: "c519ff6c-5b5f-4383-ab09-fca419176a29"). InnerVolumeSpecName "kube-api-access-8wbwp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:09:25.236934 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.236917 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c519ff6c-5b5f-4383-ab09-fca419176a29" (UID: "c519ff6c-5b5f-4383-ab09-fca419176a29"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:09:25.237130 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.237105 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-token" (OuterVolumeSpecName: "istio-token") pod "c519ff6c-5b5f-4383-ab09-fca419176a29" (UID: "c519ff6c-5b5f-4383-ab09-fca419176a29"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:09:25.335780 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.335738 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-token\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:09:25.335780 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.335766 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-podinfo\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:09:25.335780 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.335775 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-data\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:09:25.335780 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.335785 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wbwp\" (UniqueName: \"kubernetes.io/projected/c519ff6c-5b5f-4383-ab09-fca419176a29-kube-api-access-8wbwp\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:09:25.335780 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.335794 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c519ff6c-5b5f-4383-ab09-fca419176a29-istio-envoy\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:09:25.336164 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.335802 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c519ff6c-5b5f-4383-ab09-fca419176a29-istiod-ca-cert\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:09:25.804279 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.804232 2579 generic.go:356] "Generic (PLEG): container finished" podID="c519ff6c-5b5f-4383-ab09-fca419176a29" containerID="d015bfea342ae779d2792e183f261e8760370dd22dd928cffb4527144e382055" exitCode=0 Jul 24 14:09:25.804466 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.804287 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" Jul 24 14:09:25.804466 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.804319 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" event={"ID":"c519ff6c-5b5f-4383-ab09-fca419176a29","Type":"ContainerDied","Data":"d015bfea342ae779d2792e183f261e8760370dd22dd928cffb4527144e382055"} Jul 24 14:09:25.804466 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.804380 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59" event={"ID":"c519ff6c-5b5f-4383-ab09-fca419176a29","Type":"ContainerDied","Data":"d8fef1513b1b36fa73617a4f45e22437d50772c528e37731fec11689f42bed60"} Jul 24 14:09:25.804466 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.804415 2579 scope.go:117] "RemoveContainer" containerID="d015bfea342ae779d2792e183f261e8760370dd22dd928cffb4527144e382055" Jul 24 14:09:25.814139 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.814121 2579 scope.go:117] "RemoveContainer" containerID="a6118b5be68d3e7959df5be108edb1b227e98db242f70e90e9dadcb66b822b04" Jul 24 14:09:25.822780 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.822762 2579 scope.go:117] "RemoveContainer" containerID="d015bfea342ae779d2792e183f261e8760370dd22dd928cffb4527144e382055" Jul 24 14:09:25.823054 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:09:25.823030 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d015bfea342ae779d2792e183f261e8760370dd22dd928cffb4527144e382055\": container with ID starting with d015bfea342ae779d2792e183f261e8760370dd22dd928cffb4527144e382055 not found: ID does not exist" containerID="d015bfea342ae779d2792e183f261e8760370dd22dd928cffb4527144e382055" Jul 24 14:09:25.823129 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.823065 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d015bfea342ae779d2792e183f261e8760370dd22dd928cffb4527144e382055"} err="failed to get container status \"d015bfea342ae779d2792e183f261e8760370dd22dd928cffb4527144e382055\": rpc error: code = NotFound desc = could not find container \"d015bfea342ae779d2792e183f261e8760370dd22dd928cffb4527144e382055\": container with ID starting with d015bfea342ae779d2792e183f261e8760370dd22dd928cffb4527144e382055 not found: ID does not exist" Jul 24 14:09:25.823129 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.823085 2579 scope.go:117] "RemoveContainer" containerID="a6118b5be68d3e7959df5be108edb1b227e98db242f70e90e9dadcb66b822b04" Jul 24 14:09:25.823341 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:09:25.823324 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6118b5be68d3e7959df5be108edb1b227e98db242f70e90e9dadcb66b822b04\": container with ID starting with a6118b5be68d3e7959df5be108edb1b227e98db242f70e90e9dadcb66b822b04 not found: ID does not exist" containerID="a6118b5be68d3e7959df5be108edb1b227e98db242f70e90e9dadcb66b822b04" Jul 24 14:09:25.823389 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.823347 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6118b5be68d3e7959df5be108edb1b227e98db242f70e90e9dadcb66b822b04"} err="failed to get container status \"a6118b5be68d3e7959df5be108edb1b227e98db242f70e90e9dadcb66b822b04\": rpc error: code = NotFound desc = could not find container \"a6118b5be68d3e7959df5be108edb1b227e98db242f70e90e9dadcb66b822b04\": container with ID starting with a6118b5be68d3e7959df5be108edb1b227e98db242f70e90e9dadcb66b822b04 not found: ID does not exist" Jul 24 14:09:25.904122 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.904090 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59"] Jul 24 14:09:25.906433 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:25.906408 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3098bd50-openshift-default-6548d7cdd-jdn59"] Jul 24 14:09:26.420154 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:26.420123 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c519ff6c-5b5f-4383-ab09-fca419176a29" path="/var/lib/kubelet/pods/c519ff6c-5b5f-4383-ab09-fca419176a29/volumes" Jul 24 14:09:31.666866 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:31.666834 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:31.784614 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:31.784577 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:32.456246 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:32.456205 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:35.709308 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:35.709272 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:09:39.070771 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:39.070732 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:39.152258 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:39.152220 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:39.314142 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:39.314103 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:39.419777 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:39.419688 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:49.874131 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:49.874099 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:09:50.212923 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:09:50.212829 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:11.208346 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:11.208257 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:19.486958 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:19.486919 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:19.598573 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:19.598534 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:20.169522 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.169485 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:20.196305 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.196277 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs"] Jul 24 14:10:20.196653 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.196641 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c519ff6c-5b5f-4383-ab09-fca419176a29" containerName="istio-proxy" Jul 24 14:10:20.196701 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.196655 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c519ff6c-5b5f-4383-ab09-fca419176a29" containerName="istio-proxy" Jul 24 14:10:20.196736 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.196723 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c519ff6c-5b5f-4383-ab09-fca419176a29" containerName="istio-proxy" Jul 24 14:10:20.200945 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.200922 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.204680 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.204657 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c21e7d-openshift-default-dockercfg-9qd5p\"" Jul 24 14:10:20.215031 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.214981 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs"] Jul 24 14:10:20.242768 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.242734 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-workload-socket\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.242959 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.242923 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-workload-certs\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.243082 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.242973 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-credential-socket\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.243082 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.243063 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-envoy\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.243201 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.243088 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-token\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.243201 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.243136 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-podinfo\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.243313 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.243205 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb56a186-13eb-4fde-a419-a69c85a7591a-istiod-ca-cert\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.243313 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.243230 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjwz2\" (UniqueName: \"kubernetes.io/projected/eb56a186-13eb-4fde-a419-a69c85a7591a-kube-api-access-kjwz2\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.243313 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.243258 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-data\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.264444 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.264412 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:20.344240 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.344198 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-workload-socket\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.344420 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.344254 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-workload-certs\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.344420 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.344303 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-credential-socket\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.344420 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.344337 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-envoy\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.344420 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.344359 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-token\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.344634 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.344601 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-podinfo\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.344711 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.344688 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb56a186-13eb-4fde-a419-a69c85a7591a-istiod-ca-cert\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.344772 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.344720 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjwz2\" (UniqueName: \"kubernetes.io/projected/eb56a186-13eb-4fde-a419-a69c85a7591a-kube-api-access-kjwz2\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.344772 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.344740 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-workload-socket\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.344772 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.344748 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-data\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.344937 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.344830 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-workload-certs\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.345155 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.345128 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-data\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.345437 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.345370 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-credential-socket\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.345712 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.345562 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb56a186-13eb-4fde-a419-a69c85a7591a-istiod-ca-cert\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.347289 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.347260 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-podinfo\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.347405 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.347345 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-envoy\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.356856 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.356831 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjwz2\" (UniqueName: \"kubernetes.io/projected/eb56a186-13eb-4fde-a419-a69c85a7591a-kube-api-access-kjwz2\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.356946 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.356923 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-token\") pod \"e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.512361 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.512269 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:20.662278 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.662247 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs"] Jul 24 14:10:20.665657 ip-10-0-137-60 kubenswrapper[2579]: W0724 14:10:20.665626 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb56a186_13eb_4fde_a419_a69c85a7591a.slice/crio-d4cfbaafd9adbf76f369ccb69d94f8258431de618a8a404cc03641aed0f9d8cb WatchSource:0}: Error finding container d4cfbaafd9adbf76f369ccb69d94f8258431de618a8a404cc03641aed0f9d8cb: Status 404 returned error can't find the container with id d4cfbaafd9adbf76f369ccb69d94f8258431de618a8a404cc03641aed0f9d8cb Jul 24 14:10:20.667809 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.667774 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:10:20.667894 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.667844 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:10:20.667894 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.667874 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:10:20.975005 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:20.974958 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:21.115959 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:21.115919 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" event={"ID":"eb56a186-13eb-4fde-a419-a69c85a7591a","Type":"ContainerStarted","Data":"8e85fae65049c0c884e02298cf59d2f9fa0e7fcb9ab8890ac5daee038e53612c"} Jul 24 14:10:21.116236 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:21.115985 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" event={"ID":"eb56a186-13eb-4fde-a419-a69c85a7591a","Type":"ContainerStarted","Data":"d4cfbaafd9adbf76f369ccb69d94f8258431de618a8a404cc03641aed0f9d8cb"} Jul 24 14:10:21.141372 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:21.141317 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" podStartSLOduration=1.141297945 podStartE2EDuration="1.141297945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:10:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:10:21.139067546 +0000 UTC m=+2163.262449945" watchObservedRunningTime="2026-07-24 14:10:21.141297945 +0000 UTC m=+2163.264680344" Jul 24 14:10:21.513409 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:21.513366 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:21.515033 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:21.515002 2579 patch_prober.go:28] interesting pod/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 24 14:10:21.515152 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:21.515059 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" podUID="eb56a186-13eb-4fde-a419-a69c85a7591a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 24 14:10:22.517210 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:22.517172 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:23.135287 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:23.135254 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:23.136176 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:23.136151 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:23.180725 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:23.180687 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:23.272203 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:23.272170 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:23.463508 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:23.463421 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:23.819705 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:23.819674 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:24.196463 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:24.196382 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:24.297967 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:24.297935 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:26.336379 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:26.336341 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:10:27.747812 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:27.747775 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:28.037433 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.037335 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:28.411950 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.411914 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l"] Jul 24 14:10:28.412552 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.412532 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c519ff6c-5b5f-4383-ab09-fca419176a29" containerName="istio-proxy" Jul 24 14:10:28.412610 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.412558 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c519ff6c-5b5f-4383-ab09-fca419176a29" containerName="istio-proxy" Jul 24 14:10:28.412653 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.412641 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c519ff6c-5b5f-4383-ab09-fca419176a29" containerName="istio-proxy" Jul 24 14:10:28.416747 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.416709 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.423092 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.423068 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-d52a8f-openshift-default-dockercfg-s55zq\"" Jul 24 14:10:28.485765 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.485684 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:28.486225 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.486204 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l"] Jul 24 14:10:28.525603 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.525551 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-credential-socket\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.525791 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.525611 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6da0ef36-2226-40a5-8462-1556d9763cd6-istiod-ca-cert\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.525791 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.525681 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-data\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.525791 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.525720 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-envoy\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.525791 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.525786 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-podinfo\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.526104 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.525820 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-workload-certs\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.526104 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.525873 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-workload-socket\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.526104 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.525902 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-token\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.526104 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.525929 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h55mg\" (UniqueName: \"kubernetes.io/projected/6da0ef36-2226-40a5-8462-1556d9763cd6-kube-api-access-h55mg\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.627387 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.627350 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-data\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.627590 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.627398 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-envoy\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.627590 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.627422 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-podinfo\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.627590 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.627444 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-workload-certs\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.627590 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.627478 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-workload-socket\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.627590 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.627495 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-token\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.627590 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.627513 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h55mg\" (UniqueName: \"kubernetes.io/projected/6da0ef36-2226-40a5-8462-1556d9763cd6-kube-api-access-h55mg\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.627590 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.627547 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-credential-socket\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.627590 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.627564 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6da0ef36-2226-40a5-8462-1556d9763cd6-istiod-ca-cert\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.627976 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.627930 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-workload-socket\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.628148 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.628125 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-credential-socket\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.628389 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.628351 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-workload-certs\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.628477 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.628407 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-data\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.628730 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.628712 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6da0ef36-2226-40a5-8462-1556d9763cd6-istiod-ca-cert\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.630405 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.630384 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-envoy\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.630780 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.630758 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-podinfo\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.641582 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.641561 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h55mg\" (UniqueName: \"kubernetes.io/projected/6da0ef36-2226-40a5-8462-1556d9763cd6-kube-api-access-h55mg\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.646619 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.646587 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-token\") pod \"e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.709269 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.709172 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:28.728737 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.728708 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:28.936724 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.936608 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l"] Jul 24 14:10:28.937944 ip-10-0-137-60 kubenswrapper[2579]: W0724 14:10:28.937920 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6da0ef36_2226_40a5_8462_1556d9763cd6.slice/crio-239050d87d5808aa420ec4257d8621322772fc0d0ad187a2a65ff0442905e7c8 WatchSource:0}: Error finding container 239050d87d5808aa420ec4257d8621322772fc0d0ad187a2a65ff0442905e7c8: Status 404 returned error can't find the container with id 239050d87d5808aa420ec4257d8621322772fc0d0ad187a2a65ff0442905e7c8 Jul 24 14:10:28.940273 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.940182 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:10:28.940376 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.940279 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:10:28.940376 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:28.940310 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:10:29.186605 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:29.186572 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" event={"ID":"6da0ef36-2226-40a5-8462-1556d9763cd6","Type":"ContainerStarted","Data":"179ab00068a44da82b243ad139579b1f3bdeec17f6dbc32188dc43dca0d8ab92"} Jul 24 14:10:29.186781 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:29.186624 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" event={"ID":"6da0ef36-2226-40a5-8462-1556d9763cd6","Type":"ContainerStarted","Data":"239050d87d5808aa420ec4257d8621322772fc0d0ad187a2a65ff0442905e7c8"} Jul 24 14:10:29.226046 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:29.225882 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" podStartSLOduration=1.225864101 podStartE2EDuration="1.225864101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:10:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:10:29.22521686 +0000 UTC m=+2171.348599258" watchObservedRunningTime="2026-07-24 14:10:29.225864101 +0000 UTC m=+2171.349246499" Jul 24 14:10:29.730021 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:29.729951 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:30.734491 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:30.734462 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:31.200415 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:31.200385 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:31.201535 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:31.201512 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:31.288866 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:31.288716 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:31.479015 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:31.478899 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:31.601140 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:31.601110 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:31.786891 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:31.786805 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:32.117324 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:32.117289 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:32.214983 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:32.214947 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:35.227007 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:35.226959 2579 generic.go:356] "Generic (PLEG): container finished" podID="eb56a186-13eb-4fde-a419-a69c85a7591a" containerID="8e85fae65049c0c884e02298cf59d2f9fa0e7fcb9ab8890ac5daee038e53612c" exitCode=0 Jul 24 14:10:35.227440 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:35.227034 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" event={"ID":"eb56a186-13eb-4fde-a419-a69c85a7591a","Type":"ContainerDied","Data":"8e85fae65049c0c884e02298cf59d2f9fa0e7fcb9ab8890ac5daee038e53612c"} Jul 24 14:10:35.227440 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:35.227421 2579 scope.go:117] "RemoveContainer" containerID="8e85fae65049c0c884e02298cf59d2f9fa0e7fcb9ab8890ac5daee038e53612c" Jul 24 14:10:35.228453 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:35.228424 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:10:35.228551 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:35.228489 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:10:35.228551 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:35.228524 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:10:35.752594 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:35.752542 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:36.235809 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:36.235766 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" event={"ID":"eb56a186-13eb-4fde-a419-a69c85a7591a","Type":"ContainerStarted","Data":"1a0db289c4e477e4a86fa8778ab3b4a71cbf911a94d625a72a95499b1fc28f34"} Jul 24 14:10:36.513240 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:36.513159 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:36.514192 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:36.514164 2579 patch_prober.go:28] interesting pod/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 24 14:10:36.514317 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:36.514243 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" podUID="eb56a186-13eb-4fde-a419-a69c85a7591a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 24 14:10:37.513324 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:37.513285 2579 patch_prober.go:28] interesting pod/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 24 14:10:37.513721 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:37.513344 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" podUID="eb56a186-13eb-4fde-a419-a69c85a7591a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 24 14:10:38.135710 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:38.135674 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:38.513372 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:38.513275 2579 patch_prober.go:28] interesting pod/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 24 14:10:38.513372 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:38.513335 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" podUID="eb56a186-13eb-4fde-a419-a69c85a7591a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 24 14:10:39.513655 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:39.513614 2579 patch_prober.go:28] interesting pod/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 24 14:10:39.514087 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:39.513666 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" podUID="eb56a186-13eb-4fde-a419-a69c85a7591a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 24 14:10:40.516532 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:40.516500 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:40.517543 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:40.517521 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:10:44.291836 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:44.291738 2579 generic.go:356] "Generic (PLEG): container finished" podID="6da0ef36-2226-40a5-8462-1556d9763cd6" containerID="179ab00068a44da82b243ad139579b1f3bdeec17f6dbc32188dc43dca0d8ab92" exitCode=0 Jul 24 14:10:44.291836 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:44.291818 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" event={"ID":"6da0ef36-2226-40a5-8462-1556d9763cd6","Type":"ContainerDied","Data":"179ab00068a44da82b243ad139579b1f3bdeec17f6dbc32188dc43dca0d8ab92"} Jul 24 14:10:44.292327 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:44.292297 2579 scope.go:117] "RemoveContainer" containerID="179ab00068a44da82b243ad139579b1f3bdeec17f6dbc32188dc43dca0d8ab92" Jul 24 14:10:44.293348 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:44.293323 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:10:44.293413 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:44.293401 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:10:44.293461 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:44.293441 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:10:44.621739 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:44.621704 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:10:44.642949 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:44.642901 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:10:45.302189 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:45.302146 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" event={"ID":"6da0ef36-2226-40a5-8462-1556d9763cd6","Type":"ContainerStarted","Data":"a6fa819d4a8884abe6b462018c3d2dc54c2bd15d19e49bbc1be7cc35799da865"} Jul 24 14:10:45.729532 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:45.729495 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:45.730650 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:45.730623 2579 patch_prober.go:28] interesting pod/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 24 14:10:45.730779 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:45.730674 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" podUID="6da0ef36-2226-40a5-8462-1556d9763cd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 24 14:10:46.200681 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:46.200631 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:46.730274 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:46.730244 2579 patch_prober.go:28] interesting pod/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 24 14:10:46.730638 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:46.730296 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" podUID="6da0ef36-2226-40a5-8462-1556d9763cd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 24 14:10:47.730148 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:47.730109 2579 patch_prober.go:28] interesting pod/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 24 14:10:47.730367 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:47.730175 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" podUID="6da0ef36-2226-40a5-8462-1556d9763cd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 24 14:10:48.730064 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:48.730018 2579 patch_prober.go:28] interesting pod/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 24 14:10:48.730245 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:48.730083 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" podUID="6da0ef36-2226-40a5-8462-1556d9763cd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 24 14:10:49.733560 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:49.733526 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:49.734516 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:49.734498 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:10:59.905578 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:10:59.905544 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:11:10.744829 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:11:10.744782 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:11:16.213984 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:11:16.213948 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:11:19.165867 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:11:19.165831 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:11:50.437043 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:11:50.436911 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:11:51.437117 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:11:51.437075 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:11:55.233984 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:11:55.233938 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:12:02.254788 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:02.254750 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:02.343690 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:02.343658 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:02.413277 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:02.413236 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:06.668564 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:06.668531 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:07.121895 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:07.121861 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:08.029983 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:08.029944 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:12:13.445357 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:13.445318 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:13.521948 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:13.521913 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:13.731318 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:13.731235 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs"] Jul 24 14:12:13.731516 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:13.731483 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" podUID="eb56a186-13eb-4fde-a419-a69c85a7591a" containerName="istio-proxy" containerID="cri-o://1a0db289c4e477e4a86fa8778ab3b4a71cbf911a94d625a72a95499b1fc28f34" gracePeriod=30 Jul 24 14:12:13.751510 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:13.751485 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:13.967269 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:13.967234 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:18.998302 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:18.998242 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:12:19.045144 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.045106 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-data\") pod \"eb56a186-13eb-4fde-a419-a69c85a7591a\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " Jul 24 14:12:19.045318 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.045158 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-envoy\") pod \"eb56a186-13eb-4fde-a419-a69c85a7591a\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " Jul 24 14:12:19.045318 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.045208 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjwz2\" (UniqueName: \"kubernetes.io/projected/eb56a186-13eb-4fde-a419-a69c85a7591a-kube-api-access-kjwz2\") pod \"eb56a186-13eb-4fde-a419-a69c85a7591a\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " Jul 24 14:12:19.045318 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.045242 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-token\") pod \"eb56a186-13eb-4fde-a419-a69c85a7591a\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " Jul 24 14:12:19.045318 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.045274 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-podinfo\") pod \"eb56a186-13eb-4fde-a419-a69c85a7591a\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " Jul 24 14:12:19.045318 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.045317 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-workload-certs\") pod \"eb56a186-13eb-4fde-a419-a69c85a7591a\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " Jul 24 14:12:19.045617 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.045388 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb56a186-13eb-4fde-a419-a69c85a7591a-istiod-ca-cert\") pod \"eb56a186-13eb-4fde-a419-a69c85a7591a\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " Jul 24 14:12:19.045617 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.045453 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-credential-socket\") pod \"eb56a186-13eb-4fde-a419-a69c85a7591a\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " Jul 24 14:12:19.045617 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.045489 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-workload-socket\") pod \"eb56a186-13eb-4fde-a419-a69c85a7591a\" (UID: \"eb56a186-13eb-4fde-a419-a69c85a7591a\") " Jul 24 14:12:19.045837 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.045806 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eb56a186-13eb-4fde-a419-a69c85a7591a" (UID: "eb56a186-13eb-4fde-a419-a69c85a7591a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:12:19.045909 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.045882 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb56a186-13eb-4fde-a419-a69c85a7591a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eb56a186-13eb-4fde-a419-a69c85a7591a" (UID: "eb56a186-13eb-4fde-a419-a69c85a7591a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:12:19.046042 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.045983 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eb56a186-13eb-4fde-a419-a69c85a7591a" (UID: "eb56a186-13eb-4fde-a419-a69c85a7591a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:12:19.046124 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.046075 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eb56a186-13eb-4fde-a419-a69c85a7591a" (UID: "eb56a186-13eb-4fde-a419-a69c85a7591a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:12:19.046124 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.046082 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-data" (OuterVolumeSpecName: "istio-data") pod "eb56a186-13eb-4fde-a419-a69c85a7591a" (UID: "eb56a186-13eb-4fde-a419-a69c85a7591a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:12:19.047860 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.047831 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eb56a186-13eb-4fde-a419-a69c85a7591a" (UID: "eb56a186-13eb-4fde-a419-a69c85a7591a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:12:19.047978 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.047911 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eb56a186-13eb-4fde-a419-a69c85a7591a" (UID: "eb56a186-13eb-4fde-a419-a69c85a7591a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:12:19.048068 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.048029 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb56a186-13eb-4fde-a419-a69c85a7591a-kube-api-access-kjwz2" (OuterVolumeSpecName: "kube-api-access-kjwz2") pod "eb56a186-13eb-4fde-a419-a69c85a7591a" (UID: "eb56a186-13eb-4fde-a419-a69c85a7591a"). InnerVolumeSpecName "kube-api-access-kjwz2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:12:19.048068 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.048052 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-token" (OuterVolumeSpecName: "istio-token") pod "eb56a186-13eb-4fde-a419-a69c85a7591a" (UID: "eb56a186-13eb-4fde-a419-a69c85a7591a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:12:19.146889 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.146857 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-workload-certs\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:12:19.146889 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.146883 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb56a186-13eb-4fde-a419-a69c85a7591a-istiod-ca-cert\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:12:19.146889 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.146892 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-credential-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:12:19.147166 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.146900 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-workload-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:12:19.147166 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.146909 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-data\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:12:19.147166 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.146917 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-envoy\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:12:19.147166 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.146925 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjwz2\" (UniqueName: \"kubernetes.io/projected/eb56a186-13eb-4fde-a419-a69c85a7591a-kube-api-access-kjwz2\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:12:19.147166 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.146932 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-token\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:12:19.147166 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.146942 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb56a186-13eb-4fde-a419-a69c85a7591a-istio-podinfo\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:12:19.841329 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.841296 2579 generic.go:356] "Generic (PLEG): container finished" podID="eb56a186-13eb-4fde-a419-a69c85a7591a" containerID="1a0db289c4e477e4a86fa8778ab3b4a71cbf911a94d625a72a95499b1fc28f34" exitCode=0 Jul 24 14:12:19.841501 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.841349 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" Jul 24 14:12:19.841501 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.841353 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" event={"ID":"eb56a186-13eb-4fde-a419-a69c85a7591a","Type":"ContainerDied","Data":"1a0db289c4e477e4a86fa8778ab3b4a71cbf911a94d625a72a95499b1fc28f34"} Jul 24 14:12:19.841501 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.841479 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs" event={"ID":"eb56a186-13eb-4fde-a419-a69c85a7591a","Type":"ContainerDied","Data":"d4cfbaafd9adbf76f369ccb69d94f8258431de618a8a404cc03641aed0f9d8cb"} Jul 24 14:12:19.841613 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.841511 2579 scope.go:117] "RemoveContainer" containerID="1a0db289c4e477e4a86fa8778ab3b4a71cbf911a94d625a72a95499b1fc28f34" Jul 24 14:12:19.851699 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.851532 2579 scope.go:117] "RemoveContainer" containerID="8e85fae65049c0c884e02298cf59d2f9fa0e7fcb9ab8890ac5daee038e53612c" Jul 24 14:12:19.859742 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.859724 2579 scope.go:117] "RemoveContainer" containerID="1a0db289c4e477e4a86fa8778ab3b4a71cbf911a94d625a72a95499b1fc28f34" Jul 24 14:12:19.860007 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:12:19.859971 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a0db289c4e477e4a86fa8778ab3b4a71cbf911a94d625a72a95499b1fc28f34\": container with ID starting with 1a0db289c4e477e4a86fa8778ab3b4a71cbf911a94d625a72a95499b1fc28f34 not found: ID does not exist" containerID="1a0db289c4e477e4a86fa8778ab3b4a71cbf911a94d625a72a95499b1fc28f34" Jul 24 14:12:19.860064 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.860013 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a0db289c4e477e4a86fa8778ab3b4a71cbf911a94d625a72a95499b1fc28f34"} err="failed to get container status \"1a0db289c4e477e4a86fa8778ab3b4a71cbf911a94d625a72a95499b1fc28f34\": rpc error: code = NotFound desc = could not find container \"1a0db289c4e477e4a86fa8778ab3b4a71cbf911a94d625a72a95499b1fc28f34\": container with ID starting with 1a0db289c4e477e4a86fa8778ab3b4a71cbf911a94d625a72a95499b1fc28f34 not found: ID does not exist" Jul 24 14:12:19.860064 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.860030 2579 scope.go:117] "RemoveContainer" containerID="8e85fae65049c0c884e02298cf59d2f9fa0e7fcb9ab8890ac5daee038e53612c" Jul 24 14:12:19.860292 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:12:19.860273 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e85fae65049c0c884e02298cf59d2f9fa0e7fcb9ab8890ac5daee038e53612c\": container with ID starting with 8e85fae65049c0c884e02298cf59d2f9fa0e7fcb9ab8890ac5daee038e53612c not found: ID does not exist" containerID="8e85fae65049c0c884e02298cf59d2f9fa0e7fcb9ab8890ac5daee038e53612c" Jul 24 14:12:19.860331 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.860297 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e85fae65049c0c884e02298cf59d2f9fa0e7fcb9ab8890ac5daee038e53612c"} err="failed to get container status \"8e85fae65049c0c884e02298cf59d2f9fa0e7fcb9ab8890ac5daee038e53612c\": rpc error: code = NotFound desc = could not find container \"8e85fae65049c0c884e02298cf59d2f9fa0e7fcb9ab8890ac5daee038e53612c\": container with ID starting with 8e85fae65049c0c884e02298cf59d2f9fa0e7fcb9ab8890ac5daee038e53612c not found: ID does not exist" Jul 24 14:12:19.867000 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.866965 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs"] Jul 24 14:12:19.873095 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:19.873073 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c21e7d-openshift-default-c7c89c85d-6rhbs"] Jul 24 14:12:20.419898 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:20.419854 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb56a186-13eb-4fde-a419-a69c85a7591a" path="/var/lib/kubelet/pods/eb56a186-13eb-4fde-a419-a69c85a7591a/volumes" Jul 24 14:12:24.738679 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:24.738627 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:24.827372 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:24.827344 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:24.929206 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:24.929173 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:25.935733 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:25.935696 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:26.008501 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:26.008458 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:27.733238 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:27.733200 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:12:29.202005 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:29.201948 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:29.679597 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:29.679561 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:35.745195 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:35.745136 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:35.815741 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:35.815710 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:36.020273 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:36.020179 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l"] Jul 24 14:12:36.020477 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:36.020449 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" podUID="6da0ef36-2226-40a5-8462-1556d9763cd6" containerName="istio-proxy" containerID="cri-o://a6fa819d4a8884abe6b462018c3d2dc54c2bd15d19e49bbc1be7cc35799da865" gracePeriod=30 Jul 24 14:12:36.051948 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:36.051914 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:36.283050 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:36.282933 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:36.624703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:36.624674 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:36.924073 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:36.923934 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9"] Jul 24 14:12:36.924541 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:36.924521 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb56a186-13eb-4fde-a419-a69c85a7591a" containerName="istio-proxy" Jul 24 14:12:36.924595 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:36.924546 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb56a186-13eb-4fde-a419-a69c85a7591a" containerName="istio-proxy" Jul 24 14:12:36.924595 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:36.924561 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb56a186-13eb-4fde-a419-a69c85a7591a" containerName="istio-proxy" Jul 24 14:12:36.924595 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:36.924570 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb56a186-13eb-4fde-a419-a69c85a7591a" containerName="istio-proxy" Jul 24 14:12:36.924698 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:36.924635 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb56a186-13eb-4fde-a419-a69c85a7591a" containerName="istio-proxy" Jul 24 14:12:36.924698 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:36.924643 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb56a186-13eb-4fde-a419-a69c85a7591a" containerName="istio-proxy" Jul 24 14:12:36.926726 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:36.926709 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:36.930121 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:36.930094 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-41b861-openshift-default-dockercfg-fcz5l\"" Jul 24 14:12:36.943132 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:36.943106 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9"] Jul 24 14:12:36.960720 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:36.960686 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:37.029646 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.029610 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-workload-certs\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.029865 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.029670 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istiod-ca-cert\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.029865 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.029784 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-workload-socket\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.029865 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.029835 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r6vp\" (UniqueName: \"kubernetes.io/projected/1c377d67-db87-4c4a-9d75-ac1b8134c2af-kube-api-access-8r6vp\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.030082 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.029875 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-envoy\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.030082 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.029912 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-data\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.030082 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.029946 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-token\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.030082 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.030015 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-credential-socket\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.030082 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.030062 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-podinfo\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.131547 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.131482 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r6vp\" (UniqueName: \"kubernetes.io/projected/1c377d67-db87-4c4a-9d75-ac1b8134c2af-kube-api-access-8r6vp\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.131547 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.131553 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-envoy\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.131821 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.131584 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-data\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.131821 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.131611 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-token\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.131821 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.131646 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-credential-socket\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.131821 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.131683 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-podinfo\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.131821 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.131759 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-workload-certs\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.131821 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.131804 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istiod-ca-cert\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.132184 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.131862 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-workload-socket\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.132184 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.132116 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-data\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.132184 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.132142 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-credential-socket\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.132334 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.132212 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-workload-certs\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.132334 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.132251 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-workload-socket\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.132556 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.132536 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istiod-ca-cert\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.134221 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.134198 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-envoy\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.134309 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.134294 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-podinfo\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.141264 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.141237 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-token\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.141541 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.141521 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r6vp\" (UniqueName: \"kubernetes.io/projected/1c377d67-db87-4c4a-9d75-ac1b8134c2af-kube-api-access-8r6vp\") pod \"e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.238035 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.237920 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:37.391828 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.391799 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9"] Jul 24 14:12:37.392449 ip-10-0-137-60 kubenswrapper[2579]: W0724 14:12:37.392423 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c377d67_db87_4c4a_9d75_ac1b8134c2af.slice/crio-f33be76d096578e51ae658a81863911ee40fce295627a1f39e17bead2d3b7797 WatchSource:0}: Error finding container f33be76d096578e51ae658a81863911ee40fce295627a1f39e17bead2d3b7797: Status 404 returned error can't find the container with id f33be76d096578e51ae658a81863911ee40fce295627a1f39e17bead2d3b7797 Jul 24 14:12:37.394714 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.394685 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:37.394804 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.394707 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:12:37.394804 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.394770 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:12:37.394804 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.394801 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:12:37.947223 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.947185 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" event={"ID":"1c377d67-db87-4c4a-9d75-ac1b8134c2af","Type":"ContainerStarted","Data":"74d50295681e7a3a26f9ac8be56f851389f000bdba1c3f1ba3ed5faf96e4087d"} Jul 24 14:12:37.947715 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.947250 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" event={"ID":"1c377d67-db87-4c4a-9d75-ac1b8134c2af","Type":"ContainerStarted","Data":"f33be76d096578e51ae658a81863911ee40fce295627a1f39e17bead2d3b7797"} Jul 24 14:12:37.972308 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:37.972262 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" podStartSLOduration=1.972247058 podStartE2EDuration="1.972247058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:12:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:12:37.970801565 +0000 UTC m=+2300.094183964" watchObservedRunningTime="2026-07-24 14:12:37.972247058 +0000 UTC m=+2300.095629455" Jul 24 14:12:38.238382 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:38.238295 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:38.243120 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:38.243092 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:38.955264 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:38.955234 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:38.956179 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:38.956160 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:39.940231 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:39.940198 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:40.046851 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:40.046809 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:40.135614 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:40.135580 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:40.571586 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:40.571551 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:40.933505 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:40.933417 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:41.048014 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.044862 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:41.280540 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.280516 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:12:41.375135 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.375101 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-credential-socket\") pod \"6da0ef36-2226-40a5-8462-1556d9763cd6\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " Jul 24 14:12:41.375135 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.375150 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-token\") pod \"6da0ef36-2226-40a5-8462-1556d9763cd6\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " Jul 24 14:12:41.375403 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.375172 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-data\") pod \"6da0ef36-2226-40a5-8462-1556d9763cd6\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " Jul 24 14:12:41.375403 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.375192 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-envoy\") pod \"6da0ef36-2226-40a5-8462-1556d9763cd6\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " Jul 24 14:12:41.375403 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.375237 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-workload-certs\") pod \"6da0ef36-2226-40a5-8462-1556d9763cd6\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " Jul 24 14:12:41.375403 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.375280 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6da0ef36-2226-40a5-8462-1556d9763cd6-istiod-ca-cert\") pod \"6da0ef36-2226-40a5-8462-1556d9763cd6\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " Jul 24 14:12:41.375403 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.375304 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-podinfo\") pod \"6da0ef36-2226-40a5-8462-1556d9763cd6\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " Jul 24 14:12:41.375665 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.375416 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6da0ef36-2226-40a5-8462-1556d9763cd6" (UID: "6da0ef36-2226-40a5-8462-1556d9763cd6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:12:41.375665 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.375474 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-workload-socket\") pod \"6da0ef36-2226-40a5-8462-1556d9763cd6\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " Jul 24 14:12:41.375665 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.375527 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h55mg\" (UniqueName: \"kubernetes.io/projected/6da0ef36-2226-40a5-8462-1556d9763cd6-kube-api-access-h55mg\") pod \"6da0ef36-2226-40a5-8462-1556d9763cd6\" (UID: \"6da0ef36-2226-40a5-8462-1556d9763cd6\") " Jul 24 14:12:41.375665 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.375580 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6da0ef36-2226-40a5-8462-1556d9763cd6" (UID: "6da0ef36-2226-40a5-8462-1556d9763cd6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:12:41.375665 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.375641 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6da0ef36-2226-40a5-8462-1556d9763cd6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6da0ef36-2226-40a5-8462-1556d9763cd6" (UID: "6da0ef36-2226-40a5-8462-1556d9763cd6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:12:41.376129 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.375753 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6da0ef36-2226-40a5-8462-1556d9763cd6" (UID: "6da0ef36-2226-40a5-8462-1556d9763cd6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:12:41.376129 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.375921 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-workload-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:12:41.376129 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.375938 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-credential-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:12:41.376129 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.375951 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-workload-certs\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:12:41.376129 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.375963 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6da0ef36-2226-40a5-8462-1556d9763cd6-istiod-ca-cert\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:12:41.376404 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.376157 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-data" (OuterVolumeSpecName: "istio-data") pod "6da0ef36-2226-40a5-8462-1556d9763cd6" (UID: "6da0ef36-2226-40a5-8462-1556d9763cd6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:12:41.377626 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.377601 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-token" (OuterVolumeSpecName: "istio-token") pod "6da0ef36-2226-40a5-8462-1556d9763cd6" (UID: "6da0ef36-2226-40a5-8462-1556d9763cd6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:12:41.377873 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.377852 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6da0ef36-2226-40a5-8462-1556d9763cd6-kube-api-access-h55mg" (OuterVolumeSpecName: "kube-api-access-h55mg") pod "6da0ef36-2226-40a5-8462-1556d9763cd6" (UID: "6da0ef36-2226-40a5-8462-1556d9763cd6"). InnerVolumeSpecName "kube-api-access-h55mg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:12:41.378175 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.378159 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6da0ef36-2226-40a5-8462-1556d9763cd6" (UID: "6da0ef36-2226-40a5-8462-1556d9763cd6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:12:41.378228 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.378182 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6da0ef36-2226-40a5-8462-1556d9763cd6" (UID: "6da0ef36-2226-40a5-8462-1556d9763cd6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:12:41.477111 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.477022 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h55mg\" (UniqueName: \"kubernetes.io/projected/6da0ef36-2226-40a5-8462-1556d9763cd6-kube-api-access-h55mg\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:12:41.477111 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.477053 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-token\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:12:41.477111 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.477062 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-data\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:12:41.477111 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.477071 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-envoy\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:12:41.477111 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.477079 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6da0ef36-2226-40a5-8462-1556d9763cd6-istio-podinfo\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:12:41.976403 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.976374 2579 generic.go:356] "Generic (PLEG): container finished" podID="6da0ef36-2226-40a5-8462-1556d9763cd6" containerID="a6fa819d4a8884abe6b462018c3d2dc54c2bd15d19e49bbc1be7cc35799da865" exitCode=0 Jul 24 14:12:41.976583 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.976435 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" Jul 24 14:12:41.976583 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.976448 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" event={"ID":"6da0ef36-2226-40a5-8462-1556d9763cd6","Type":"ContainerDied","Data":"a6fa819d4a8884abe6b462018c3d2dc54c2bd15d19e49bbc1be7cc35799da865"} Jul 24 14:12:41.976583 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.976492 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l" event={"ID":"6da0ef36-2226-40a5-8462-1556d9763cd6","Type":"ContainerDied","Data":"239050d87d5808aa420ec4257d8621322772fc0d0ad187a2a65ff0442905e7c8"} Jul 24 14:12:41.976583 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.976513 2579 scope.go:117] "RemoveContainer" containerID="a6fa819d4a8884abe6b462018c3d2dc54c2bd15d19e49bbc1be7cc35799da865" Jul 24 14:12:41.987319 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.987300 2579 scope.go:117] "RemoveContainer" containerID="179ab00068a44da82b243ad139579b1f3bdeec17f6dbc32188dc43dca0d8ab92" Jul 24 14:12:41.995844 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.995820 2579 scope.go:117] "RemoveContainer" containerID="a6fa819d4a8884abe6b462018c3d2dc54c2bd15d19e49bbc1be7cc35799da865" Jul 24 14:12:41.996141 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:12:41.996120 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6fa819d4a8884abe6b462018c3d2dc54c2bd15d19e49bbc1be7cc35799da865\": container with ID starting with a6fa819d4a8884abe6b462018c3d2dc54c2bd15d19e49bbc1be7cc35799da865 not found: ID does not exist" containerID="a6fa819d4a8884abe6b462018c3d2dc54c2bd15d19e49bbc1be7cc35799da865" Jul 24 14:12:41.996214 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.996149 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6fa819d4a8884abe6b462018c3d2dc54c2bd15d19e49bbc1be7cc35799da865"} err="failed to get container status \"a6fa819d4a8884abe6b462018c3d2dc54c2bd15d19e49bbc1be7cc35799da865\": rpc error: code = NotFound desc = could not find container \"a6fa819d4a8884abe6b462018c3d2dc54c2bd15d19e49bbc1be7cc35799da865\": container with ID starting with a6fa819d4a8884abe6b462018c3d2dc54c2bd15d19e49bbc1be7cc35799da865 not found: ID does not exist" Jul 24 14:12:41.996214 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.996167 2579 scope.go:117] "RemoveContainer" containerID="179ab00068a44da82b243ad139579b1f3bdeec17f6dbc32188dc43dca0d8ab92" Jul 24 14:12:41.996421 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:12:41.996403 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"179ab00068a44da82b243ad139579b1f3bdeec17f6dbc32188dc43dca0d8ab92\": container with ID starting with 179ab00068a44da82b243ad139579b1f3bdeec17f6dbc32188dc43dca0d8ab92 not found: ID does not exist" containerID="179ab00068a44da82b243ad139579b1f3bdeec17f6dbc32188dc43dca0d8ab92" Jul 24 14:12:41.996486 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:41.996430 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"179ab00068a44da82b243ad139579b1f3bdeec17f6dbc32188dc43dca0d8ab92"} err="failed to get container status \"179ab00068a44da82b243ad139579b1f3bdeec17f6dbc32188dc43dca0d8ab92\": rpc error: code = NotFound desc = could not find container \"179ab00068a44da82b243ad139579b1f3bdeec17f6dbc32188dc43dca0d8ab92\": container with ID starting with 179ab00068a44da82b243ad139579b1f3bdeec17f6dbc32188dc43dca0d8ab92 not found: ID does not exist" Jul 24 14:12:42.001227 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:42.001203 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l"] Jul 24 14:12:42.005763 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:42.005737 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-d52a8f-openshift-default-5b79bb9576-mtl7l"] Jul 24 14:12:42.419692 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:42.419648 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6da0ef36-2226-40a5-8462-1556d9763cd6" path="/var/lib/kubelet/pods/6da0ef36-2226-40a5-8462-1556d9763cd6/volumes" Jul 24 14:12:44.456095 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:44.456056 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:44.954396 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:44.954361 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:45.255220 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.255143 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq"] Jul 24 14:12:45.255530 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.255515 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6da0ef36-2226-40a5-8462-1556d9763cd6" containerName="istio-proxy" Jul 24 14:12:45.255576 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.255531 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6da0ef36-2226-40a5-8462-1556d9763cd6" containerName="istio-proxy" Jul 24 14:12:45.255576 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.255551 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6da0ef36-2226-40a5-8462-1556d9763cd6" containerName="istio-proxy" Jul 24 14:12:45.255576 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.255557 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6da0ef36-2226-40a5-8462-1556d9763cd6" containerName="istio-proxy" Jul 24 14:12:45.255668 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.255634 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6da0ef36-2226-40a5-8462-1556d9763cd6" containerName="istio-proxy" Jul 24 14:12:45.258819 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.258804 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.261916 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.261894 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-17b50a-openshift-default-dockercfg-mkzlb\"" Jul 24 14:12:45.272637 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.272613 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq"] Jul 24 14:12:45.291120 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.291079 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:45.413830 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.413785 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-credential-socket\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.413830 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.413832 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-data\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.414140 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.413911 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-workload-socket\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.414140 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.413951 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-envoy\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.414140 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.413977 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-podinfo\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.414140 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.414068 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istiod-ca-cert\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.414283 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.414143 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-token\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.414283 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.414197 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22499\" (UniqueName: \"kubernetes.io/projected/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-kube-api-access-22499\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.414283 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.414228 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-workload-certs\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.477071 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.477034 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:45.515043 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.514916 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-envoy\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.515043 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.514962 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-podinfo\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.515296 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.515045 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istiod-ca-cert\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.515296 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.515190 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-token\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.515296 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.515263 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22499\" (UniqueName: \"kubernetes.io/projected/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-kube-api-access-22499\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.515296 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.515287 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-workload-certs\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.515521 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.515319 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-credential-socket\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.515521 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.515348 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-data\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.515521 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.515410 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-workload-socket\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.515679 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.515618 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-workload-certs\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.515819 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.515797 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-workload-socket\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.515896 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.515820 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-credential-socket\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.515896 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.515875 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-data\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.516090 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.516071 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istiod-ca-cert\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.518120 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.518090 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-envoy\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.518315 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.518297 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-podinfo\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.525273 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.525228 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-token\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.525407 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.525336 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22499\" (UniqueName: \"kubernetes.io/projected/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-kube-api-access-22499\") pod \"e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.570329 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.570285 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:45.726054 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.725158 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq"] Jul 24 14:12:45.730338 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.730299 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:12:45.730456 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.730382 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:12:45.730456 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:45.730419 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:12:46.000292 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:46.000255 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" event={"ID":"015b16a8-c7f6-4d6a-ab17-0701b5703e3e","Type":"ContainerStarted","Data":"976edd3dc115dcb22f86076125dabf733deac6fa43440a52cd19c2b04e09b223"} Jul 24 14:12:46.000472 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:46.000320 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" event={"ID":"015b16a8-c7f6-4d6a-ab17-0701b5703e3e","Type":"ContainerStarted","Data":"14333bd648a20c6a31928359e63796b4900f6841f8ee6a99b37247863d629e2f"} Jul 24 14:12:46.029418 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:46.029305 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" podStartSLOduration=1.029287248 podStartE2EDuration="1.029287248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:12:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:12:46.026631342 +0000 UTC m=+2308.150013743" watchObservedRunningTime="2026-07-24 14:12:46.029287248 +0000 UTC m=+2308.152669646" Jul 24 14:12:46.570732 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:46.570693 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:46.575767 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:46.575741 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:47.006391 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:47.006307 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:47.007186 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:47.007170 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:12:47.801537 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:47.801500 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:12:48.031148 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:48.031111 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:48.125116 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:48.125079 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:48.206890 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:48.206856 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:48.757592 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:48.757497 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:48.848964 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:48.848895 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:52.512210 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:52.512173 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:54.997240 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:54.997151 2579 patch_prober.go:28] interesting pod/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 14:12:54.997668 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:54.997269 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" podUID="1c377d67-db87-4c4a-9d75-ac1b8134c2af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 14:12:56.061823 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:56.061784 2579 generic.go:356] "Generic (PLEG): container finished" podID="1c377d67-db87-4c4a-9d75-ac1b8134c2af" containerID="74d50295681e7a3a26f9ac8be56f851389f000bdba1c3f1ba3ed5faf96e4087d" exitCode=0 Jul 24 14:12:56.062343 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:56.061859 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" event={"ID":"1c377d67-db87-4c4a-9d75-ac1b8134c2af","Type":"ContainerDied","Data":"74d50295681e7a3a26f9ac8be56f851389f000bdba1c3f1ba3ed5faf96e4087d"} Jul 24 14:12:56.062343 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:56.062288 2579 scope.go:117] "RemoveContainer" containerID="74d50295681e7a3a26f9ac8be56f851389f000bdba1c3f1ba3ed5faf96e4087d" Jul 24 14:12:56.063302 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:56.063270 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:12:56.063400 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:56.063341 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:12:56.063400 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:56.063374 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:12:57.073476 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:57.073441 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" event={"ID":"1c377d67-db87-4c4a-9d75-ac1b8134c2af","Type":"ContainerStarted","Data":"7e31e5609b80dd214181859a6b39d6590c76b1e0ba1164df86d5733c6909aa8f"} Jul 24 14:12:57.238251 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:57.238211 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:12:57.238425 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:57.238398 2579 patch_prober.go:28] interesting pod/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 24 14:12:57.238471 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:57.238448 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" podUID="1c377d67-db87-4c4a-9d75-ac1b8134c2af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 24 14:12:58.238335 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:58.238295 2579 patch_prober.go:28] interesting pod/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 24 14:12:58.238726 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:58.238355 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" podUID="1c377d67-db87-4c4a-9d75-ac1b8134c2af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 24 14:12:59.239366 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:59.239327 2579 patch_prober.go:28] interesting pod/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 24 14:12:59.239764 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:12:59.239386 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" podUID="1c377d67-db87-4c4a-9d75-ac1b8134c2af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 24 14:13:00.238766 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:00.238730 2579 patch_prober.go:28] interesting pod/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 24 14:13:00.238951 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:00.238782 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" podUID="1c377d67-db87-4c4a-9d75-ac1b8134c2af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 24 14:13:01.242012 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:01.241971 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:13:01.243122 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:01.243098 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:13:01.248540 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:01.248522 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:13:04.108275 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:04.108241 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:13:04.517804 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:04.517710 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:05.127865 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:05.127776 2579 generic.go:356] "Generic (PLEG): container finished" podID="015b16a8-c7f6-4d6a-ab17-0701b5703e3e" containerID="976edd3dc115dcb22f86076125dabf733deac6fa43440a52cd19c2b04e09b223" exitCode=0 Jul 24 14:13:05.127865 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:05.127848 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" event={"ID":"015b16a8-c7f6-4d6a-ab17-0701b5703e3e","Type":"ContainerDied","Data":"976edd3dc115dcb22f86076125dabf733deac6fa43440a52cd19c2b04e09b223"} Jul 24 14:13:05.128467 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:05.128399 2579 scope.go:117] "RemoveContainer" containerID="976edd3dc115dcb22f86076125dabf733deac6fa43440a52cd19c2b04e09b223" Jul 24 14:13:05.129622 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:05.129592 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:13:05.129701 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:05.129680 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:13:05.129752 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:05.129722 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:13:06.135794 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:06.135759 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" event={"ID":"015b16a8-c7f6-4d6a-ab17-0701b5703e3e","Type":"ContainerStarted","Data":"8db2e0a3251229f28ef9f819703fdaa88d441061c987517e2a0d07bcff4737d9"} Jul 24 14:13:06.571495 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:06.571459 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:13:06.571801 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:06.571772 2579 patch_prober.go:28] interesting pod/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 24 14:13:06.571872 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:06.571849 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" podUID="015b16a8-c7f6-4d6a-ab17-0701b5703e3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 24 14:13:07.570679 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:07.570635 2579 patch_prober.go:28] interesting pod/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 24 14:13:07.571254 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:07.570698 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" podUID="015b16a8-c7f6-4d6a-ab17-0701b5703e3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 24 14:13:08.571539 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:08.571490 2579 patch_prober.go:28] interesting pod/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 24 14:13:08.572047 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:08.571560 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" podUID="015b16a8-c7f6-4d6a-ab17-0701b5703e3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 24 14:13:09.571449 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:09.571397 2579 patch_prober.go:28] interesting pod/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 24 14:13:09.571684 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:09.571470 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" podUID="015b16a8-c7f6-4d6a-ab17-0701b5703e3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 24 14:13:10.574512 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:10.574480 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:13:10.575015 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:10.574881 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:13:10.575521 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:10.575506 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:13:16.062339 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:16.062306 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:16.174939 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:16.174901 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:16.366264 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:16.366173 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:18.528659 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:18.528620 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:20.844358 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:20.844319 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:21.024909 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:21.024871 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:13:21.274961 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:21.274871 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:23.762899 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:23.762869 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:27.824380 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:27.824342 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:27.924332 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:27.924287 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:28.103366 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:28.103271 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9"] Jul 24 14:13:28.103551 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:28.103522 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" podUID="1c377d67-db87-4c4a-9d75-ac1b8134c2af" containerName="istio-proxy" containerID="cri-o://7e31e5609b80dd214181859a6b39d6590c76b1e0ba1164df86d5733c6909aa8f" gracePeriod=30 Jul 24 14:13:28.129497 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:28.129467 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:28.380366 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:28.380286 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:31.245253 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:31.245219 2579 patch_prober.go:28] interesting pod/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 14:13:31.245755 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:31.245278 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" podUID="1c377d67-db87-4c4a-9d75-ac1b8134c2af" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 14:13:33.309200 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.308615 2579 generic.go:356] "Generic (PLEG): container finished" podID="1c377d67-db87-4c4a-9d75-ac1b8134c2af" containerID="7e31e5609b80dd214181859a6b39d6590c76b1e0ba1164df86d5733c6909aa8f" exitCode=0 Jul 24 14:13:33.309200 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.308812 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" event={"ID":"1c377d67-db87-4c4a-9d75-ac1b8134c2af","Type":"ContainerDied","Data":"7e31e5609b80dd214181859a6b39d6590c76b1e0ba1164df86d5733c6909aa8f"} Jul 24 14:13:33.309200 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.308869 2579 scope.go:117] "RemoveContainer" containerID="74d50295681e7a3a26f9ac8be56f851389f000bdba1c3f1ba3ed5faf96e4087d" Jul 24 14:13:33.389974 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.389949 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:13:33.511061 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.511019 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-data\") pod \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " Jul 24 14:13:33.511250 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.511070 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-workload-socket\") pod \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " Jul 24 14:13:33.511250 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.511099 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-credential-socket\") pod \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " Jul 24 14:13:33.511250 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.511121 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-token\") pod \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " Jul 24 14:13:33.511250 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.511139 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-envoy\") pod \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " Jul 24 14:13:33.511250 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.511196 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-workload-certs\") pod \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " Jul 24 14:13:33.511517 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.511261 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istiod-ca-cert\") pod \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " Jul 24 14:13:33.511517 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.511306 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-podinfo\") pod \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " Jul 24 14:13:33.511517 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.511331 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8r6vp\" (UniqueName: \"kubernetes.io/projected/1c377d67-db87-4c4a-9d75-ac1b8134c2af-kube-api-access-8r6vp\") pod \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\" (UID: \"1c377d67-db87-4c4a-9d75-ac1b8134c2af\") " Jul 24 14:13:33.511517 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.511389 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1c377d67-db87-4c4a-9d75-ac1b8134c2af" (UID: "1c377d67-db87-4c4a-9d75-ac1b8134c2af"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:13:33.511728 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.511637 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1c377d67-db87-4c4a-9d75-ac1b8134c2af" (UID: "1c377d67-db87-4c4a-9d75-ac1b8134c2af"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:13:33.511728 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.511648 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1c377d67-db87-4c4a-9d75-ac1b8134c2af" (UID: "1c377d67-db87-4c4a-9d75-ac1b8134c2af"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:13:33.511831 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.511763 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1c377d67-db87-4c4a-9d75-ac1b8134c2af" (UID: "1c377d67-db87-4c4a-9d75-ac1b8134c2af"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:13:33.511896 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.511839 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-workload-certs\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:13:33.511896 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.511859 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istiod-ca-cert\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:13:33.511896 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.511872 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-workload-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:13:33.511896 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.511870 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-data" (OuterVolumeSpecName: "istio-data") pod "1c377d67-db87-4c4a-9d75-ac1b8134c2af" (UID: "1c377d67-db87-4c4a-9d75-ac1b8134c2af"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:13:33.511896 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.511889 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-credential-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:13:33.513844 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.513798 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1c377d67-db87-4c4a-9d75-ac1b8134c2af" (UID: "1c377d67-db87-4c4a-9d75-ac1b8134c2af"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:13:33.513967 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.513881 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1c377d67-db87-4c4a-9d75-ac1b8134c2af" (UID: "1c377d67-db87-4c4a-9d75-ac1b8134c2af"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:13:33.514205 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.514184 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-token" (OuterVolumeSpecName: "istio-token") pod "1c377d67-db87-4c4a-9d75-ac1b8134c2af" (UID: "1c377d67-db87-4c4a-9d75-ac1b8134c2af"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:13:33.514315 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.514295 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c377d67-db87-4c4a-9d75-ac1b8134c2af-kube-api-access-8r6vp" (OuterVolumeSpecName: "kube-api-access-8r6vp") pod "1c377d67-db87-4c4a-9d75-ac1b8134c2af" (UID: "1c377d67-db87-4c4a-9d75-ac1b8134c2af"). InnerVolumeSpecName "kube-api-access-8r6vp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:13:33.612895 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.612864 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-token\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:13:33.612895 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.612894 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-envoy\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:13:33.612895 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.612903 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-podinfo\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:13:33.613158 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.612912 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r6vp\" (UniqueName: \"kubernetes.io/projected/1c377d67-db87-4c4a-9d75-ac1b8134c2af-kube-api-access-8r6vp\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:13:33.613158 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:33.612920 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c377d67-db87-4c4a-9d75-ac1b8134c2af-istio-data\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:13:34.315907 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:34.315876 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" Jul 24 14:13:34.315907 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:34.315896 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9" event={"ID":"1c377d67-db87-4c4a-9d75-ac1b8134c2af","Type":"ContainerDied","Data":"f33be76d096578e51ae658a81863911ee40fce295627a1f39e17bead2d3b7797"} Jul 24 14:13:34.316410 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:34.315949 2579 scope.go:117] "RemoveContainer" containerID="7e31e5609b80dd214181859a6b39d6590c76b1e0ba1164df86d5733c6909aa8f" Jul 24 14:13:34.346180 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:34.346151 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9"] Jul 24 14:13:34.352940 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:34.352917 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-41b861-openshift-default-fc6f897bd-72xf9"] Jul 24 14:13:34.420155 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:34.420127 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c377d67-db87-4c4a-9d75-ac1b8134c2af" path="/var/lib/kubelet/pods/1c377d67-db87-4c4a-9d75-ac1b8134c2af/volumes" Jul 24 14:13:44.297240 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:44.297209 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:44.520046 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:44.520011 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:44.640261 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:44.640227 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:48.591122 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:48.591085 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:48.813108 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:48.813069 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:13:48.990122 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:48.990037 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:55.105927 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:55.105888 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:55.183959 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:55.183931 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:55.366602 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:55.366520 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq"] Jul 24 14:13:55.366768 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:55.366736 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" podUID="015b16a8-c7f6-4d6a-ab17-0701b5703e3e" containerName="istio-proxy" containerID="cri-o://8db2e0a3251229f28ef9f819703fdaa88d441061c987517e2a0d07bcff4737d9" gracePeriod=30 Jul 24 14:13:55.393345 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:55.393312 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:55.577444 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:55.577405 2579 patch_prober.go:28] interesting pod/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 24 14:13:55.577660 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:55.577489 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" podUID="015b16a8-c7f6-4d6a-ab17-0701b5703e3e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 14:13:55.734838 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:55.734751 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:56.037024 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.036903 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:56.388681 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.388647 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c"] Jul 24 14:13:56.389133 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.389117 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c377d67-db87-4c4a-9d75-ac1b8134c2af" containerName="istio-proxy" Jul 24 14:13:56.389187 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.389135 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c377d67-db87-4c4a-9d75-ac1b8134c2af" containerName="istio-proxy" Jul 24 14:13:56.389223 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.389213 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6da0ef36-2226-40a5-8462-1556d9763cd6" containerName="istio-proxy" Jul 24 14:13:56.389223 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.389221 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c377d67-db87-4c4a-9d75-ac1b8134c2af" containerName="istio-proxy" Jul 24 14:13:56.389287 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.389227 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c377d67-db87-4c4a-9d75-ac1b8134c2af" containerName="istio-proxy" Jul 24 14:13:56.391525 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.391511 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.394633 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.394614 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-4cd364-openshift-default-dockercfg-6zptc\"" Jul 24 14:13:56.413053 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.413016 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c"] Jul 24 14:13:56.419475 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.419442 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:56.436183 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.436153 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-token\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.436356 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.436188 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-podinfo\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.436356 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.436217 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-workload-socket\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.436356 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.436273 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-data\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.436501 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.436434 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-credential-socket\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.436501 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.436459 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-envoy\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.436595 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.436502 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-workload-certs\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.436595 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.436517 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgdgn\" (UniqueName: \"kubernetes.io/projected/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-kube-api-access-sgdgn\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.436706 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.436687 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istiod-ca-cert\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.537985 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.537942 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-workload-certs\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.538153 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.537986 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgdgn\" (UniqueName: \"kubernetes.io/projected/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-kube-api-access-sgdgn\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.538153 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.538085 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istiod-ca-cert\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.538153 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.538137 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-token\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.538337 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.538172 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-podinfo\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.538337 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.538197 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-workload-socket\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.538337 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.538234 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-data\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.538337 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.538305 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-credential-socket\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.538337 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.538330 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-envoy\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.538621 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.538593 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-workload-socket\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.538685 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.538605 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-data\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.538746 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.538693 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-workload-certs\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.538746 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.538695 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-credential-socket\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.538842 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.538819 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istiod-ca-cert\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.540655 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.540632 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-envoy\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.540934 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.540918 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-podinfo\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.547645 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.547620 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-token\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.547779 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.547718 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgdgn\" (UniqueName: \"kubernetes.io/projected/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-kube-api-access-sgdgn\") pod \"e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.701601 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.701511 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:56.726899 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.726854 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:56.855554 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.855522 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c"] Jul 24 14:13:56.863533 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.863494 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:13:56.863662 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.863594 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:13:56.863717 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:56.863678 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:13:57.455558 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:57.455519 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" event={"ID":"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8","Type":"ContainerStarted","Data":"52f77fe2ade8fdc0cdbc76784d6db9c2f86a4c145523c0937b1c0622518d2b82"} Jul 24 14:13:57.456090 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:57.455579 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" event={"ID":"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8","Type":"ContainerStarted","Data":"d52a348004fe992756b632aa455313fcb481277db39427bf4a3504b7cfd1a5f2"} Jul 24 14:13:57.482426 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:57.482372 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" podStartSLOduration=1.482356121 podStartE2EDuration="1.482356121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:13:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:13:57.48038439 +0000 UTC m=+2379.603766789" watchObservedRunningTime="2026-07-24 14:13:57.482356121 +0000 UTC m=+2379.605738519" Jul 24 14:13:57.701709 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:57.701673 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:57.707112 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:57.707038 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:58.461305 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:58.461270 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:58.462318 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:58.462301 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:13:59.476045 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:59.476015 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:59.573619 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:59.573589 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:59.647278 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:59.647237 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:13:59.842219 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:13:59.842184 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:14:00.308333 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.308293 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:14:00.401187 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.401161 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:14:00.625157 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.625134 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:14:00.680271 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.680244 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-workload-certs\") pod \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " Jul 24 14:14:00.680456 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.680297 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22499\" (UniqueName: \"kubernetes.io/projected/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-kube-api-access-22499\") pod \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " Jul 24 14:14:00.680456 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.680363 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-podinfo\") pod \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " Jul 24 14:14:00.680456 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.680411 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-token\") pod \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " Jul 24 14:14:00.680456 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.680442 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-envoy\") pod \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " Jul 24 14:14:00.680687 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.680474 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-credential-socket\") pod \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " Jul 24 14:14:00.680687 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.680546 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-workload-socket\") pod \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " Jul 24 14:14:00.680687 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.680543 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "015b16a8-c7f6-4d6a-ab17-0701b5703e3e" (UID: "015b16a8-c7f6-4d6a-ab17-0701b5703e3e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:14:00.680687 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.680580 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-data\") pod \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " Jul 24 14:14:00.680687 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.680602 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istiod-ca-cert\") pod \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\" (UID: \"015b16a8-c7f6-4d6a-ab17-0701b5703e3e\") " Jul 24 14:14:00.680945 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.680832 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "015b16a8-c7f6-4d6a-ab17-0701b5703e3e" (UID: "015b16a8-c7f6-4d6a-ab17-0701b5703e3e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:14:00.681059 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.681037 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-workload-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:14:00.681123 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.681065 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-workload-certs\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:14:00.681252 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.681141 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "015b16a8-c7f6-4d6a-ab17-0701b5703e3e" (UID: "015b16a8-c7f6-4d6a-ab17-0701b5703e3e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:14:00.681404 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.681363 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-data" (OuterVolumeSpecName: "istio-data") pod "015b16a8-c7f6-4d6a-ab17-0701b5703e3e" (UID: "015b16a8-c7f6-4d6a-ab17-0701b5703e3e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:14:00.681404 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.681372 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "015b16a8-c7f6-4d6a-ab17-0701b5703e3e" (UID: "015b16a8-c7f6-4d6a-ab17-0701b5703e3e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:14:00.682965 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.682933 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-kube-api-access-22499" (OuterVolumeSpecName: "kube-api-access-22499") pod "015b16a8-c7f6-4d6a-ab17-0701b5703e3e" (UID: "015b16a8-c7f6-4d6a-ab17-0701b5703e3e"). InnerVolumeSpecName "kube-api-access-22499". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:14:00.683110 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.683054 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-token" (OuterVolumeSpecName: "istio-token") pod "015b16a8-c7f6-4d6a-ab17-0701b5703e3e" (UID: "015b16a8-c7f6-4d6a-ab17-0701b5703e3e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:14:00.683178 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.683137 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "015b16a8-c7f6-4d6a-ab17-0701b5703e3e" (UID: "015b16a8-c7f6-4d6a-ab17-0701b5703e3e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:14:00.683247 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.683226 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "015b16a8-c7f6-4d6a-ab17-0701b5703e3e" (UID: "015b16a8-c7f6-4d6a-ab17-0701b5703e3e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:14:00.782163 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.782124 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-token\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:14:00.782163 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.782155 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-envoy\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:14:00.782163 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.782169 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-credential-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:14:00.782396 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.782181 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-data\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:14:00.782396 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.782194 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istiod-ca-cert\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:14:00.782396 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.782204 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22499\" (UniqueName: \"kubernetes.io/projected/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-kube-api-access-22499\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:14:00.782396 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:00.782214 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/015b16a8-c7f6-4d6a-ab17-0701b5703e3e-istio-podinfo\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:14:01.481085 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:01.481045 2579 generic.go:356] "Generic (PLEG): container finished" podID="015b16a8-c7f6-4d6a-ab17-0701b5703e3e" containerID="8db2e0a3251229f28ef9f819703fdaa88d441061c987517e2a0d07bcff4737d9" exitCode=0 Jul 24 14:14:01.481281 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:01.481092 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" Jul 24 14:14:01.481281 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:01.481131 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" event={"ID":"015b16a8-c7f6-4d6a-ab17-0701b5703e3e","Type":"ContainerDied","Data":"8db2e0a3251229f28ef9f819703fdaa88d441061c987517e2a0d07bcff4737d9"} Jul 24 14:14:01.481281 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:01.481187 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq" event={"ID":"015b16a8-c7f6-4d6a-ab17-0701b5703e3e","Type":"ContainerDied","Data":"14333bd648a20c6a31928359e63796b4900f6841f8ee6a99b37247863d629e2f"} Jul 24 14:14:01.481281 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:01.481211 2579 scope.go:117] "RemoveContainer" containerID="8db2e0a3251229f28ef9f819703fdaa88d441061c987517e2a0d07bcff4737d9" Jul 24 14:14:01.491062 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:01.491042 2579 scope.go:117] "RemoveContainer" containerID="976edd3dc115dcb22f86076125dabf733deac6fa43440a52cd19c2b04e09b223" Jul 24 14:14:01.499366 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:01.499347 2579 scope.go:117] "RemoveContainer" containerID="8db2e0a3251229f28ef9f819703fdaa88d441061c987517e2a0d07bcff4737d9" Jul 24 14:14:01.499608 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:14:01.499590 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8db2e0a3251229f28ef9f819703fdaa88d441061c987517e2a0d07bcff4737d9\": container with ID starting with 8db2e0a3251229f28ef9f819703fdaa88d441061c987517e2a0d07bcff4737d9 not found: ID does not exist" containerID="8db2e0a3251229f28ef9f819703fdaa88d441061c987517e2a0d07bcff4737d9" Jul 24 14:14:01.499661 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:01.499619 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8db2e0a3251229f28ef9f819703fdaa88d441061c987517e2a0d07bcff4737d9"} err="failed to get container status \"8db2e0a3251229f28ef9f819703fdaa88d441061c987517e2a0d07bcff4737d9\": rpc error: code = NotFound desc = could not find container \"8db2e0a3251229f28ef9f819703fdaa88d441061c987517e2a0d07bcff4737d9\": container with ID starting with 8db2e0a3251229f28ef9f819703fdaa88d441061c987517e2a0d07bcff4737d9 not found: ID does not exist" Jul 24 14:14:01.499661 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:01.499635 2579 scope.go:117] "RemoveContainer" containerID="976edd3dc115dcb22f86076125dabf733deac6fa43440a52cd19c2b04e09b223" Jul 24 14:14:01.499874 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:14:01.499857 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"976edd3dc115dcb22f86076125dabf733deac6fa43440a52cd19c2b04e09b223\": container with ID starting with 976edd3dc115dcb22f86076125dabf733deac6fa43440a52cd19c2b04e09b223 not found: ID does not exist" containerID="976edd3dc115dcb22f86076125dabf733deac6fa43440a52cd19c2b04e09b223" Jul 24 14:14:01.499929 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:01.499880 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"976edd3dc115dcb22f86076125dabf733deac6fa43440a52cd19c2b04e09b223"} err="failed to get container status \"976edd3dc115dcb22f86076125dabf733deac6fa43440a52cd19c2b04e09b223\": rpc error: code = NotFound desc = could not find container \"976edd3dc115dcb22f86076125dabf733deac6fa43440a52cd19c2b04e09b223\": container with ID starting with 976edd3dc115dcb22f86076125dabf733deac6fa43440a52cd19c2b04e09b223 not found: ID does not exist" Jul 24 14:14:01.518438 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:01.518408 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq"] Jul 24 14:14:01.532211 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:01.532181 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-17b50a-openshift-default-bc7c5df78-5mbhq"] Jul 24 14:14:02.421196 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:02.421159 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="015b16a8-c7f6-4d6a-ab17-0701b5703e3e" path="/var/lib/kubelet/pods/015b16a8-c7f6-4d6a-ab17-0701b5703e3e/volumes" Jul 24 14:14:03.994908 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:03.994874 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:14:04.359858 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.359826 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:14:04.670968 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.670879 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k"] Jul 24 14:14:04.671307 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.671291 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c377d67-db87-4c4a-9d75-ac1b8134c2af" containerName="istio-proxy" Jul 24 14:14:04.671376 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.671308 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c377d67-db87-4c4a-9d75-ac1b8134c2af" containerName="istio-proxy" Jul 24 14:14:04.671376 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.671324 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="015b16a8-c7f6-4d6a-ab17-0701b5703e3e" containerName="istio-proxy" Jul 24 14:14:04.671376 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.671329 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="015b16a8-c7f6-4d6a-ab17-0701b5703e3e" containerName="istio-proxy" Jul 24 14:14:04.671376 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.671337 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="015b16a8-c7f6-4d6a-ab17-0701b5703e3e" containerName="istio-proxy" Jul 24 14:14:04.671376 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.671344 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="015b16a8-c7f6-4d6a-ab17-0701b5703e3e" containerName="istio-proxy" Jul 24 14:14:04.671651 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.671412 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="015b16a8-c7f6-4d6a-ab17-0701b5703e3e" containerName="istio-proxy" Jul 24 14:14:04.671651 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.671419 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="015b16a8-c7f6-4d6a-ab17-0701b5703e3e" containerName="istio-proxy" Jul 24 14:14:04.674767 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.674746 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.679511 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.679491 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-525009-openshift-default-dockercfg-mlfms\"" Jul 24 14:14:04.691556 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.691536 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k"] Jul 24 14:14:04.714359 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.714321 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:14:04.717782 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.717755 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-credential-socket\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.717900 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.717807 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-workload-certs\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.717900 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.717840 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-envoy\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.717900 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.717866 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istiod-ca-cert\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.717900 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.717889 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-podinfo\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.718096 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.717983 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z4zz\" (UniqueName: \"kubernetes.io/projected/1f2f414b-9b83-4d8d-8ef3-4582121f3532-kube-api-access-7z4zz\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.718096 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.718049 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-workload-socket\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.718096 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.718072 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-token\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.718096 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.718089 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-data\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.818844 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.818810 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z4zz\" (UniqueName: \"kubernetes.io/projected/1f2f414b-9b83-4d8d-8ef3-4582121f3532-kube-api-access-7z4zz\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.819069 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.818851 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-workload-socket\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.819069 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.818876 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-token\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.819069 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.818892 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-data\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.819243 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.819097 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-credential-socket\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.819243 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.819175 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-workload-certs\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.819243 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.819213 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-envoy\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.819393 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.819241 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-data\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.819393 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.819252 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istiod-ca-cert\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.819393 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.819297 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-podinfo\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.819393 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.819294 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-workload-socket\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.819605 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.819476 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-credential-socket\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.819605 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.819538 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-workload-certs\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.819900 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.819880 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istiod-ca-cert\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.821563 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.821543 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-envoy\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.821829 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.821810 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-podinfo\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.825472 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.825443 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:14:04.828225 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.828194 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z4zz\" (UniqueName: \"kubernetes.io/projected/1f2f414b-9b83-4d8d-8ef3-4582121f3532-kube-api-access-7z4zz\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.828315 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.828244 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-token\") pod \"e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:04.985475 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:04.985363 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:05.133724 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:05.133694 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k"] Jul 24 14:14:05.134855 ip-10-0-137-60 kubenswrapper[2579]: W0724 14:14:05.134829 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f2f414b_9b83_4d8d_8ef3_4582121f3532.slice/crio-b9950aa9f74a7e302ca181163111178e390145eb60bcfb95d60f19d5b530e0fe WatchSource:0}: Error finding container b9950aa9f74a7e302ca181163111178e390145eb60bcfb95d60f19d5b530e0fe: Status 404 returned error can't find the container with id b9950aa9f74a7e302ca181163111178e390145eb60bcfb95d60f19d5b530e0fe Jul 24 14:14:05.137599 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:05.137569 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:14:05.137672 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:05.137634 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:14:05.137672 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:05.137668 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:14:05.310370 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:05.310339 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:14:05.505453 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:05.505422 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" event={"ID":"1f2f414b-9b83-4d8d-8ef3-4582121f3532","Type":"ContainerStarted","Data":"d9c5f878bb54c39b56f9730f96b47d491ab68567f34a91051f600bd0d672572a"} Jul 24 14:14:05.505641 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:05.505476 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" event={"ID":"1f2f414b-9b83-4d8d-8ef3-4582121f3532","Type":"ContainerStarted","Data":"b9950aa9f74a7e302ca181163111178e390145eb60bcfb95d60f19d5b530e0fe"} Jul 24 14:14:05.540167 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:05.540107 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" podStartSLOduration=1.5400844089999999 podStartE2EDuration="1.540084409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:14:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:14:05.539143224 +0000 UTC m=+2387.662525648" watchObservedRunningTime="2026-07-24 14:14:05.540084409 +0000 UTC m=+2387.663466808" Jul 24 14:14:05.986224 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:05.986184 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:06.990962 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:06.990931 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:07.516419 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:07.516387 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:07.517492 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:07.517467 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:07.552622 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:07.552587 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:14:07.705887 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:07.705857 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:14:08.183603 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:08.183572 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:14:08.521876 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:08.521783 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:14:08.991097 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:08.991061 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:14:09.087932 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:09.087892 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:14:12.050006 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:12.049961 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:14:13.593082 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:13.592529 2579 patch_prober.go:28] interesting pod/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": read tcp 10.133.0.2:52994->10.133.0.55:15021: read: connection reset by peer" start-of-body= Jul 24 14:14:13.593082 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:13.592610 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" podUID="6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": read tcp 10.133.0.2:52994->10.133.0.55:15021: read: connection reset by peer" Jul 24 14:14:14.559296 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:14.559257 2579 generic.go:356] "Generic (PLEG): container finished" podID="6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" containerID="52f77fe2ade8fdc0cdbc76784d6db9c2f86a4c145523c0937b1c0622518d2b82" exitCode=0 Jul 24 14:14:14.559467 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:14.559336 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" event={"ID":"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8","Type":"ContainerDied","Data":"52f77fe2ade8fdc0cdbc76784d6db9c2f86a4c145523c0937b1c0622518d2b82"} Jul 24 14:14:14.559786 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:14.559772 2579 scope.go:117] "RemoveContainer" containerID="52f77fe2ade8fdc0cdbc76784d6db9c2f86a4c145523c0937b1c0622518d2b82" Jul 24 14:14:14.560957 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:14.560925 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:14:14.561043 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:14.561028 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:14:14.561082 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:14.561064 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:14:14.906375 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:14.906276 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:14:15.567368 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:15.567334 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" event={"ID":"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8","Type":"ContainerStarted","Data":"7f78e970a1adc5b6fd211552af665fe1584921b27ae6eb7d4ee9c46e746139cb"} Jul 24 14:14:15.702242 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:15.702212 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:14:15.702435 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:15.702409 2579 patch_prober.go:28] interesting pod/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 24 14:14:15.702516 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:15.702466 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" podUID="6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 24 14:14:16.702447 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:16.702408 2579 patch_prober.go:28] interesting pod/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 24 14:14:16.702866 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:16.702479 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" podUID="6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 24 14:14:17.702639 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:17.702595 2579 patch_prober.go:28] interesting pod/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 24 14:14:17.703095 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:17.702652 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" podUID="6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 24 14:14:18.702644 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:18.702599 2579 patch_prober.go:28] interesting pod/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 24 14:14:18.703092 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:18.702703 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" podUID="6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 24 14:14:19.701851 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:19.701812 2579 patch_prober.go:28] interesting pod/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 24 14:14:19.702067 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:19.701867 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" podUID="6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 24 14:14:20.705441 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:20.705411 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:14:20.705857 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:20.705698 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:14:20.706385 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:20.706366 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:14:23.269757 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:23.269723 2579 patch_prober.go:28] interesting pod/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": read tcp 10.133.0.2:33580->10.133.0.56:15021: read: connection reset by peer" start-of-body= Jul 24 14:14:23.270177 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:23.269834 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" podUID="1f2f414b-9b83-4d8d-8ef3-4582121f3532" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": read tcp 10.133.0.2:33580->10.133.0.56:15021: read: connection reset by peer" Jul 24 14:14:23.484582 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:23.484502 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:14:23.629335 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:23.629296 2579 generic.go:356] "Generic (PLEG): container finished" podID="1f2f414b-9b83-4d8d-8ef3-4582121f3532" containerID="d9c5f878bb54c39b56f9730f96b47d491ab68567f34a91051f600bd0d672572a" exitCode=0 Jul 24 14:14:23.629543 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:23.629350 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" event={"ID":"1f2f414b-9b83-4d8d-8ef3-4582121f3532","Type":"ContainerDied","Data":"d9c5f878bb54c39b56f9730f96b47d491ab68567f34a91051f600bd0d672572a"} Jul 24 14:14:23.629824 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:23.629806 2579 scope.go:117] "RemoveContainer" containerID="d9c5f878bb54c39b56f9730f96b47d491ab68567f34a91051f600bd0d672572a" Jul 24 14:14:23.630968 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:23.630939 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:14:23.631067 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:23.631051 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:14:23.631106 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:23.631090 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:14:24.637508 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:24.637472 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" event={"ID":"1f2f414b-9b83-4d8d-8ef3-4582121f3532","Type":"ContainerStarted","Data":"ad2b09a6d7e7b14ec9db57dcb0d9b2ddd6d8f2409aa09db89b22d3b767d832ea"} Jul 24 14:14:24.985874 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:24.985788 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:24.986042 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:24.986008 2579 patch_prober.go:28] interesting pod/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 24 14:14:24.986091 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:24.986060 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" podUID="1f2f414b-9b83-4d8d-8ef3-4582121f3532" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 24 14:14:25.985770 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:25.985732 2579 patch_prober.go:28] interesting pod/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 24 14:14:25.986209 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:25.985806 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" podUID="1f2f414b-9b83-4d8d-8ef3-4582121f3532" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 24 14:14:26.914310 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:26.914269 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:14:26.985707 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:26.985669 2579 patch_prober.go:28] interesting pod/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 24 14:14:26.985866 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:26.985728 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" podUID="1f2f414b-9b83-4d8d-8ef3-4582121f3532" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 24 14:14:27.986600 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:27.986561 2579 patch_prober.go:28] interesting pod/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 24 14:14:27.987099 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:27.986625 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" podUID="1f2f414b-9b83-4d8d-8ef3-4582121f3532" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 24 14:14:28.990040 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:28.990008 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:28.990537 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:28.990410 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:28.991365 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:28.991333 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:14:52.411746 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:52.411664 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:14:54.911232 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:14:54.911195 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:15:16.517487 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:15:16.517448 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:15:18.366601 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:15:18.366563 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:15:18.431296 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:15:18.431257 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:15:30.403742 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:15:30.403698 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:15:50.216469 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:15:50.216422 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:16:09.397750 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:16:09.397665 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:16:28.727892 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:16:28.727848 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:16:46.604089 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:16:46.604052 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:17:02.535453 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:02.535419 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:04.359778 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:04.359742 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:07.330230 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:07.330192 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:17:07.947921 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:07.947884 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk"] Jul 24 14:17:07.951132 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:07.951114 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:07.954523 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:07.954494 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4cd364\"/\"kube-root-ca.crt\"" Jul 24 14:17:07.956269 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:07.956249 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4cd364\"/\"default-dockercfg-hlqnt\"" Jul 24 14:17:07.956354 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:07.956293 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4cd364\"/\"openshift-service-ca.crt\"" Jul 24 14:17:07.956401 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:07.956345 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4cd364\"/\"sub-test-model-4cd364-kserve-self-signed-certs\"" Jul 24 14:17:07.965639 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:07.965611 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk"] Jul 24 14:17:07.970817 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:07.970773 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:08.050555 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.050513 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-dshm\") pod \"sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.050555 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.050558 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-home\") pod \"sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.050809 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.050627 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-tls-certs\") pod \"sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.050809 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.050685 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-model-cache\") pod \"sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.050809 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.050750 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-tmp-dir\") pod \"sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.050809 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.050777 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzcs8\" (UniqueName: \"kubernetes.io/projected/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-kube-api-access-pzcs8\") pod \"sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.151920 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.151881 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-model-cache\") pod \"sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.152101 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.151962 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-tmp-dir\") pod \"sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.152101 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.151981 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzcs8\" (UniqueName: \"kubernetes.io/projected/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-kube-api-access-pzcs8\") pod \"sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.152101 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.152056 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-dshm\") pod \"sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.152101 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.152089 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-home\") pod \"sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.152321 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.152135 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-tls-certs\") pod \"sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.152381 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.152359 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-model-cache\") pod \"sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.152437 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.152384 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-tmp-dir\") pod \"sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.152437 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.152419 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-home\") pod \"sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.154549 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.154511 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-dshm\") pod \"sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.154751 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.154733 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-tls-certs\") pod \"sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.170955 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.170929 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzcs8\" (UniqueName: \"kubernetes.io/projected/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-kube-api-access-pzcs8\") pod \"sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.261628 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.261537 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:08.416376 ip-10-0-137-60 kubenswrapper[2579]: W0724 14:17:08.416347 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b7630c9_d897_4c8b_93e3_a1cb5e3b5351.slice/crio-5ae1ece96b379fd676d884ff4b10ed95076acc0ddbaef417cab75e8928471c5a WatchSource:0}: Error finding container 5ae1ece96b379fd676d884ff4b10ed95076acc0ddbaef417cab75e8928471c5a: Status 404 returned error can't find the container with id 5ae1ece96b379fd676d884ff4b10ed95076acc0ddbaef417cab75e8928471c5a Jul 24 14:17:08.421145 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.421122 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk"] Jul 24 14:17:08.572227 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.572187 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" event={"ID":"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351","Type":"ContainerStarted","Data":"882c1efa45292ca5fbc2c0edd4deba88737f0572725c40cca340de21db9147b0"} Jul 24 14:17:08.572227 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:08.572236 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" event={"ID":"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351","Type":"ContainerStarted","Data":"5ae1ece96b379fd676d884ff4b10ed95076acc0ddbaef417cab75e8928471c5a"} Jul 24 14:17:18.261945 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:18.261909 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:18.262442 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:18.261969 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:18.274676 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:18.274648 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:18.304802 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:18.304748 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" podStartSLOduration=11.304733259 podStartE2EDuration="11.304733259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:17:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:17:08.60172297 +0000 UTC m=+2570.725105368" watchObservedRunningTime="2026-07-24 14:17:18.304733259 +0000 UTC m=+2580.428115653" Jul 24 14:17:18.637588 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:18.637552 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:17:21.617825 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:21.617791 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:17:33.603434 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:33.603387 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:36.823830 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:36.823795 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:40.654111 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:17:40.654022 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:18:03.622155 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:03.622116 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:18:05.570104 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:05.570054 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:21.818218 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:21.818175 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:18:28.636298 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:28.636267 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk"] Jul 24 14:18:28.666547 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:28.666513 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:29.636667 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:29.636630 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:41.214052 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:41.214013 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:18:43.656770 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:43.656725 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" podUID="3b7630c9-d897-4c8b-93e3-a1cb5e3b5351" containerName="main" containerID="cri-o://882c1efa45292ca5fbc2c0edd4deba88737f0572725c40cca340de21db9147b0" gracePeriod=45 Jul 24 14:18:43.915368 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:43.915304 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:18:44.030116 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.030082 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-tmp-dir\") pod \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " Jul 24 14:18:44.030116 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.030123 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-tls-certs\") pod \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " Jul 24 14:18:44.030344 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.030154 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-home\") pod \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " Jul 24 14:18:44.030344 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.030175 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-model-cache\") pod \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " Jul 24 14:18:44.030344 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.030260 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-dshm\") pod \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " Jul 24 14:18:44.030344 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.030298 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzcs8\" (UniqueName: \"kubernetes.io/projected/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-kube-api-access-pzcs8\") pod \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\" (UID: \"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351\") " Jul 24 14:18:44.030559 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.030457 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3b7630c9-d897-4c8b-93e3-a1cb5e3b5351" (UID: "3b7630c9-d897-4c8b-93e3-a1cb5e3b5351"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:18:44.030559 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.030472 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-home" (OuterVolumeSpecName: "home") pod "3b7630c9-d897-4c8b-93e3-a1cb5e3b5351" (UID: "3b7630c9-d897-4c8b-93e3-a1cb5e3b5351"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:18:44.030559 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.030496 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-model-cache" (OuterVolumeSpecName: "model-cache") pod "3b7630c9-d897-4c8b-93e3-a1cb5e3b5351" (UID: "3b7630c9-d897-4c8b-93e3-a1cb5e3b5351"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:18:44.030690 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.030643 2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-tmp-dir\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:18:44.030690 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.030659 2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-home\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:18:44.030690 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.030673 2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-model-cache\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:18:44.032615 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.032575 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-dshm" (OuterVolumeSpecName: "dshm") pod "3b7630c9-d897-4c8b-93e3-a1cb5e3b5351" (UID: "3b7630c9-d897-4c8b-93e3-a1cb5e3b5351"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:18:44.032712 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.032666 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-kube-api-access-pzcs8" (OuterVolumeSpecName: "kube-api-access-pzcs8") pod "3b7630c9-d897-4c8b-93e3-a1cb5e3b5351" (UID: "3b7630c9-d897-4c8b-93e3-a1cb5e3b5351"). InnerVolumeSpecName "kube-api-access-pzcs8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:18:44.032712 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.032679 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3b7630c9-d897-4c8b-93e3-a1cb5e3b5351" (UID: "3b7630c9-d897-4c8b-93e3-a1cb5e3b5351"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 14:18:44.119130 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.119098 2579 generic.go:356] "Generic (PLEG): container finished" podID="3b7630c9-d897-4c8b-93e3-a1cb5e3b5351" containerID="882c1efa45292ca5fbc2c0edd4deba88737f0572725c40cca340de21db9147b0" exitCode=0 Jul 24 14:18:44.119335 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.119140 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" Jul 24 14:18:44.119335 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.119184 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" event={"ID":"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351","Type":"ContainerDied","Data":"882c1efa45292ca5fbc2c0edd4deba88737f0572725c40cca340de21db9147b0"} Jul 24 14:18:44.119335 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.119227 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk" event={"ID":"3b7630c9-d897-4c8b-93e3-a1cb5e3b5351","Type":"ContainerDied","Data":"5ae1ece96b379fd676d884ff4b10ed95076acc0ddbaef417cab75e8928471c5a"} Jul 24 14:18:44.119335 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.119247 2579 scope.go:117] "RemoveContainer" containerID="882c1efa45292ca5fbc2c0edd4deba88737f0572725c40cca340de21db9147b0" Jul 24 14:18:44.129445 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.129427 2579 scope.go:117] "RemoveContainer" containerID="882c1efa45292ca5fbc2c0edd4deba88737f0572725c40cca340de21db9147b0" Jul 24 14:18:44.129767 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:18:44.129748 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"882c1efa45292ca5fbc2c0edd4deba88737f0572725c40cca340de21db9147b0\": container with ID starting with 882c1efa45292ca5fbc2c0edd4deba88737f0572725c40cca340de21db9147b0 not found: ID does not exist" containerID="882c1efa45292ca5fbc2c0edd4deba88737f0572725c40cca340de21db9147b0" Jul 24 14:18:44.129817 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.129778 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"882c1efa45292ca5fbc2c0edd4deba88737f0572725c40cca340de21db9147b0"} err="failed to get container status \"882c1efa45292ca5fbc2c0edd4deba88737f0572725c40cca340de21db9147b0\": rpc error: code = NotFound desc = could not find container \"882c1efa45292ca5fbc2c0edd4deba88737f0572725c40cca340de21db9147b0\": container with ID starting with 882c1efa45292ca5fbc2c0edd4deba88737f0572725c40cca340de21db9147b0 not found: ID does not exist" Jul 24 14:18:44.131248 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.131226 2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-dshm\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:18:44.131337 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.131252 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzcs8\" (UniqueName: \"kubernetes.io/projected/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-kube-api-access-pzcs8\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:18:44.131337 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.131269 2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351-tls-certs\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:18:44.153864 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.153822 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk"] Jul 24 14:18:44.166693 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.166619 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-4cd364/sub-test-model-4cd364-kserve-6648bf7fc9-mhrjk"] Jul 24 14:18:44.420761 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:44.420673 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b7630c9-d897-4c8b-93e3-a1cb5e3b5351" path="/var/lib/kubelet/pods/3b7630c9-d897-4c8b-93e3-a1cb5e3b5351/volumes" Jul 24 14:18:47.308966 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:47.308917 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:51.269322 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:18:51.269289 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:19:07.932914 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:07.932881 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:08.932862 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:08.932775 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l"] Jul 24 14:19:08.933260 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:08.933245 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b7630c9-d897-4c8b-93e3-a1cb5e3b5351" containerName="main" Jul 24 14:19:08.933515 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:08.933263 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b7630c9-d897-4c8b-93e3-a1cb5e3b5351" containerName="main" Jul 24 14:19:08.933515 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:08.933325 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b7630c9-d897-4c8b-93e3-a1cb5e3b5351" containerName="main" Jul 24 14:19:08.936532 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:08.936514 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:08.940025 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:08.939980 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-525009\"/\"openshift-service-ca.crt\"" Jul 24 14:19:08.941734 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:08.941715 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-525009\"/\"default-dockercfg-qfbwm\"" Jul 24 14:19:08.941844 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:08.941742 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-525009\"/\"rate-test-model-525009-kserve-self-signed-certs\"" Jul 24 14:19:08.941844 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:08.941724 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-525009\"/\"kube-root-ca.crt\"" Jul 24 14:19:08.949758 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:08.949733 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l"] Jul 24 14:19:08.991561 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:08.991519 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-dshm\") pod \"rate-test-model-525009-kserve-6cc7454659-b2x8l\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:08.991744 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:08.991637 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-tmp-dir\") pod \"rate-test-model-525009-kserve-6cc7454659-b2x8l\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:08.991744 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:08.991673 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-home\") pod \"rate-test-model-525009-kserve-6cc7454659-b2x8l\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:08.991744 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:08.991697 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw8lk\" (UniqueName: \"kubernetes.io/projected/e610d991-642b-4366-9558-7eb353900057-kube-api-access-hw8lk\") pod \"rate-test-model-525009-kserve-6cc7454659-b2x8l\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:08.991868 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:08.991805 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e610d991-642b-4366-9558-7eb353900057-tls-certs\") pod \"rate-test-model-525009-kserve-6cc7454659-b2x8l\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:08.991868 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:08.991844 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-model-cache\") pod \"rate-test-model-525009-kserve-6cc7454659-b2x8l\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:09.061678 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:09.061604 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:09.092730 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:09.092687 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-tmp-dir\") pod \"rate-test-model-525009-kserve-6cc7454659-b2x8l\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:09.092928 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:09.092742 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-home\") pod \"rate-test-model-525009-kserve-6cc7454659-b2x8l\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:09.092928 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:09.092781 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw8lk\" (UniqueName: \"kubernetes.io/projected/e610d991-642b-4366-9558-7eb353900057-kube-api-access-hw8lk\") pod \"rate-test-model-525009-kserve-6cc7454659-b2x8l\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:09.092928 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:09.092862 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e610d991-642b-4366-9558-7eb353900057-tls-certs\") pod \"rate-test-model-525009-kserve-6cc7454659-b2x8l\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:09.092928 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:09.092901 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-model-cache\") pod \"rate-test-model-525009-kserve-6cc7454659-b2x8l\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:09.092928 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:09.092923 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-dshm\") pod \"rate-test-model-525009-kserve-6cc7454659-b2x8l\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:09.093217 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:09.093139 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-tmp-dir\") pod \"rate-test-model-525009-kserve-6cc7454659-b2x8l\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:09.093217 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:09.093202 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-home\") pod \"rate-test-model-525009-kserve-6cc7454659-b2x8l\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:09.093330 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:09.093230 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-model-cache\") pod \"rate-test-model-525009-kserve-6cc7454659-b2x8l\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:09.095414 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:09.095390 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-dshm\") pod \"rate-test-model-525009-kserve-6cc7454659-b2x8l\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:09.095758 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:09.095739 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e610d991-642b-4366-9558-7eb353900057-tls-certs\") pod \"rate-test-model-525009-kserve-6cc7454659-b2x8l\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:09.102009 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:09.101965 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw8lk\" (UniqueName: \"kubernetes.io/projected/e610d991-642b-4366-9558-7eb353900057-kube-api-access-hw8lk\") pod \"rate-test-model-525009-kserve-6cc7454659-b2x8l\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:09.246291 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:09.246184 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:09.395475 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:09.395443 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l"] Jul 24 14:19:09.396742 ip-10-0-137-60 kubenswrapper[2579]: W0724 14:19:09.396717 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode610d991_642b_4366_9558_7eb353900057.slice/crio-e3078fe8b758183ef3afcde584711423e00783cce005ccc8962a6aa431f74d87 WatchSource:0}: Error finding container e3078fe8b758183ef3afcde584711423e00783cce005ccc8962a6aa431f74d87: Status 404 returned error can't find the container with id e3078fe8b758183ef3afcde584711423e00783cce005ccc8962a6aa431f74d87 Jul 24 14:19:10.273125 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:10.273089 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" event={"ID":"e610d991-642b-4366-9558-7eb353900057","Type":"ContainerStarted","Data":"884ac6a9889eebd22e38f56c63a6ff059ecd5342a836ce9402761fd6773b2dd8"} Jul 24 14:19:10.273548 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:10.273139 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" event={"ID":"e610d991-642b-4366-9558-7eb353900057","Type":"ContainerStarted","Data":"e3078fe8b758183ef3afcde584711423e00783cce005ccc8962a6aa431f74d87"} Jul 24 14:19:10.296850 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:10.296801 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" podStartSLOduration=2.296777054 podStartE2EDuration="2.296777054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:19:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:19:10.295427566 +0000 UTC m=+2692.418809966" watchObservedRunningTime="2026-07-24 14:19:10.296777054 +0000 UTC m=+2692.420159452" Jul 24 14:19:13.018183 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:13.018150 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:19:19.247340 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:19.247301 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:19.247823 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:19.247620 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:19.260583 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:19.260554 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:19.335175 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:19.335146 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:19:34.610125 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:34.610090 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:19:51.004617 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:51.004577 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:19:57.822808 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:57.822771 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:58.745853 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:58.745824 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l"] Jul 24 14:19:58.773660 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:19:58.773621 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:11.705960 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:11.705928 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:20:13.765635 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:13.765588 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" podUID="e610d991-642b-4366-9558-7eb353900057" containerName="main" containerID="cri-o://884ac6a9889eebd22e38f56c63a6ff059ecd5342a836ce9402761fd6773b2dd8" gracePeriod=45 Jul 24 14:20:14.026975 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.026910 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:20:14.112289 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.112253 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw8lk\" (UniqueName: \"kubernetes.io/projected/e610d991-642b-4366-9558-7eb353900057-kube-api-access-hw8lk\") pod \"e610d991-642b-4366-9558-7eb353900057\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " Jul 24 14:20:14.112465 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.112341 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-model-cache\") pod \"e610d991-642b-4366-9558-7eb353900057\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " Jul 24 14:20:14.112465 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.112369 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-dshm\") pod \"e610d991-642b-4366-9558-7eb353900057\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " Jul 24 14:20:14.112465 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.112412 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e610d991-642b-4366-9558-7eb353900057-tls-certs\") pod \"e610d991-642b-4366-9558-7eb353900057\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " Jul 24 14:20:14.112465 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.112436 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-home\") pod \"e610d991-642b-4366-9558-7eb353900057\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " Jul 24 14:20:14.112663 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.112638 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-tmp-dir\") pod \"e610d991-642b-4366-9558-7eb353900057\" (UID: \"e610d991-642b-4366-9558-7eb353900057\") " Jul 24 14:20:14.112755 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.112622 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-model-cache" (OuterVolumeSpecName: "model-cache") pod "e610d991-642b-4366-9558-7eb353900057" (UID: "e610d991-642b-4366-9558-7eb353900057"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:20:14.112824 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.112698 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-home" (OuterVolumeSpecName: "home") pod "e610d991-642b-4366-9558-7eb353900057" (UID: "e610d991-642b-4366-9558-7eb353900057"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:20:14.112879 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.112816 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e610d991-642b-4366-9558-7eb353900057" (UID: "e610d991-642b-4366-9558-7eb353900057"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:20:14.113078 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.113042 2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-tmp-dir\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:20:14.113078 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.113063 2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-model-cache\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:20:14.113078 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.113076 2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-home\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:20:14.114771 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.114750 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-dshm" (OuterVolumeSpecName: "dshm") pod "e610d991-642b-4366-9558-7eb353900057" (UID: "e610d991-642b-4366-9558-7eb353900057"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:20:14.114833 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.114805 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e610d991-642b-4366-9558-7eb353900057-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e610d991-642b-4366-9558-7eb353900057" (UID: "e610d991-642b-4366-9558-7eb353900057"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 14:20:14.114881 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.114849 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e610d991-642b-4366-9558-7eb353900057-kube-api-access-hw8lk" (OuterVolumeSpecName: "kube-api-access-hw8lk") pod "e610d991-642b-4366-9558-7eb353900057" (UID: "e610d991-642b-4366-9558-7eb353900057"). InnerVolumeSpecName "kube-api-access-hw8lk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:20:14.214509 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.214470 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw8lk\" (UniqueName: \"kubernetes.io/projected/e610d991-642b-4366-9558-7eb353900057-kube-api-access-hw8lk\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:20:14.214509 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.214505 2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e610d991-642b-4366-9558-7eb353900057-dshm\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:20:14.214509 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.214516 2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e610d991-642b-4366-9558-7eb353900057-tls-certs\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:20:14.638504 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.638468 2579 generic.go:356] "Generic (PLEG): container finished" podID="e610d991-642b-4366-9558-7eb353900057" containerID="884ac6a9889eebd22e38f56c63a6ff059ecd5342a836ce9402761fd6773b2dd8" exitCode=0 Jul 24 14:20:14.638681 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.638512 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" Jul 24 14:20:14.638681 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.638554 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" event={"ID":"e610d991-642b-4366-9558-7eb353900057","Type":"ContainerDied","Data":"884ac6a9889eebd22e38f56c63a6ff059ecd5342a836ce9402761fd6773b2dd8"} Jul 24 14:20:14.638681 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.638600 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l" event={"ID":"e610d991-642b-4366-9558-7eb353900057","Type":"ContainerDied","Data":"e3078fe8b758183ef3afcde584711423e00783cce005ccc8962a6aa431f74d87"} Jul 24 14:20:14.638681 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.638624 2579 scope.go:117] "RemoveContainer" containerID="884ac6a9889eebd22e38f56c63a6ff059ecd5342a836ce9402761fd6773b2dd8" Jul 24 14:20:14.649484 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.649461 2579 scope.go:117] "RemoveContainer" containerID="884ac6a9889eebd22e38f56c63a6ff059ecd5342a836ce9402761fd6773b2dd8" Jul 24 14:20:14.649789 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:20:14.649762 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"884ac6a9889eebd22e38f56c63a6ff059ecd5342a836ce9402761fd6773b2dd8\": container with ID starting with 884ac6a9889eebd22e38f56c63a6ff059ecd5342a836ce9402761fd6773b2dd8 not found: ID does not exist" containerID="884ac6a9889eebd22e38f56c63a6ff059ecd5342a836ce9402761fd6773b2dd8" Jul 24 14:20:14.649882 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.649800 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"884ac6a9889eebd22e38f56c63a6ff059ecd5342a836ce9402761fd6773b2dd8"} err="failed to get container status \"884ac6a9889eebd22e38f56c63a6ff059ecd5342a836ce9402761fd6773b2dd8\": rpc error: code = NotFound desc = could not find container \"884ac6a9889eebd22e38f56c63a6ff059ecd5342a836ce9402761fd6773b2dd8\": container with ID starting with 884ac6a9889eebd22e38f56c63a6ff059ecd5342a836ce9402761fd6773b2dd8 not found: ID does not exist" Jul 24 14:20:14.667774 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.667726 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l"] Jul 24 14:20:14.669663 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:14.669635 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-525009/rate-test-model-525009-kserve-6cc7454659-b2x8l"] Jul 24 14:20:16.420349 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:16.420314 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e610d991-642b-4366-9558-7eb353900057" path="/var/lib/kubelet/pods/e610d991-642b-4366-9558-7eb353900057/volumes" Jul 24 14:20:18.689220 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:18.689185 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:19.001543 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.001450 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx"] Jul 24 14:20:19.001962 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.001942 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e610d991-642b-4366-9558-7eb353900057" containerName="main" Jul 24 14:20:19.002086 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.001966 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e610d991-642b-4366-9558-7eb353900057" containerName="main" Jul 24 14:20:19.002140 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.002106 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e610d991-642b-4366-9558-7eb353900057" containerName="main" Jul 24 14:20:19.006464 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.006442 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.014960 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.014936 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-546fcb-openshift-default-dockercfg-gf8nd\"" Jul 24 14:20:19.033689 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.033668 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx"] Jul 24 14:20:19.035115 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.035094 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:19.062474 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.062437 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t49ps\" (UniqueName: \"kubernetes.io/projected/b2d16e5a-7726-4c8d-b2a9-e266da469b11-kube-api-access-t49ps\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.062632 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.062525 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-workload-certs\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.062632 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.062553 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-workload-socket\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.062632 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.062590 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-podinfo\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.062736 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.062652 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-token\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.062736 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.062684 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istiod-ca-cert\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.062736 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.062705 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-data\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.062829 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.062770 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-credential-socket\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.062829 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.062815 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-envoy\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.164023 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.163952 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-envoy\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.164023 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.164025 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t49ps\" (UniqueName: \"kubernetes.io/projected/b2d16e5a-7726-4c8d-b2a9-e266da469b11-kube-api-access-t49ps\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.164275 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.164118 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-workload-certs\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.164275 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.164160 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-workload-socket\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.164275 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.164208 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-podinfo\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.164275 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.164244 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-token\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.164491 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.164279 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istiod-ca-cert\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.164491 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.164310 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-data\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.164491 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.164346 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-credential-socket\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.164650 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.164575 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-workload-certs\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.164650 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.164628 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-workload-socket\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.164741 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.164710 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-data\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.164741 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.164733 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-credential-socket\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.165127 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.165109 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istiod-ca-cert\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.166809 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.166791 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-envoy\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.166912 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.166895 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-podinfo\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.173611 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.173583 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t49ps\" (UniqueName: \"kubernetes.io/projected/b2d16e5a-7726-4c8d-b2a9-e266da469b11-kube-api-access-t49ps\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.173734 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.173668 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-token\") pod \"e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.297662 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.297574 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:19.317195 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.317161 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:19.479494 ip-10-0-137-60 kubenswrapper[2579]: W0724 14:20:19.479462 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d16e5a_7726_4c8d_b2a9_e266da469b11.slice/crio-7ad0da9bc5239bfeba4696c19512e91031eb7a88393c44def70474fe41a7fdea WatchSource:0}: Error finding container 7ad0da9bc5239bfeba4696c19512e91031eb7a88393c44def70474fe41a7fdea: Status 404 returned error can't find the container with id 7ad0da9bc5239bfeba4696c19512e91031eb7a88393c44def70474fe41a7fdea Jul 24 14:20:19.480557 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.480535 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx"] Jul 24 14:20:19.482125 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.482078 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:20:19.482202 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.482182 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:20:19.482243 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.482228 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:20:19.680574 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.680534 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" event={"ID":"b2d16e5a-7726-4c8d-b2a9-e266da469b11","Type":"ContainerStarted","Data":"aa9dfa5e637f602222a8a919f89ef8140bc3b55ae28298c661eaf832404e88d2"} Jul 24 14:20:19.680804 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.680633 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" event={"ID":"b2d16e5a-7726-4c8d-b2a9-e266da469b11","Type":"ContainerStarted","Data":"7ad0da9bc5239bfeba4696c19512e91031eb7a88393c44def70474fe41a7fdea"} Jul 24 14:20:19.706389 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:19.706327 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" podStartSLOduration=1.70631098 podStartE2EDuration="1.70631098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:20:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:20:19.703064717 +0000 UTC m=+2761.826447116" watchObservedRunningTime="2026-07-24 14:20:19.70631098 +0000 UTC m=+2761.829693378" Jul 24 14:20:20.318217 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:20.318174 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:20.323267 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:20.323240 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:20.686128 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:20.686046 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:20.687146 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:20.687130 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:21.783787 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:21.783736 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:21.932721 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:21.932690 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:23.751542 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:23.751506 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:23.867176 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:23.867146 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:24.296582 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:24.296550 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:24.396522 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:24.396489 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:26.266532 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:26.266489 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:27.878100 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:27.878068 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:30.163725 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:30.163688 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:20:36.729188 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:36.729132 2579 patch_prober.go:28] interesting pod/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 14:20:36.729683 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:36.729207 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" podUID="b2d16e5a-7726-4c8d-b2a9-e266da469b11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 14:20:38.802437 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:38.802298 2579 generic.go:356] "Generic (PLEG): container finished" podID="b2d16e5a-7726-4c8d-b2a9-e266da469b11" containerID="aa9dfa5e637f602222a8a919f89ef8140bc3b55ae28298c661eaf832404e88d2" exitCode=0 Jul 24 14:20:38.802437 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:38.802370 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" event={"ID":"b2d16e5a-7726-4c8d-b2a9-e266da469b11","Type":"ContainerDied","Data":"aa9dfa5e637f602222a8a919f89ef8140bc3b55ae28298c661eaf832404e88d2"} Jul 24 14:20:38.802979 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:38.802830 2579 scope.go:117] "RemoveContainer" containerID="aa9dfa5e637f602222a8a919f89ef8140bc3b55ae28298c661eaf832404e88d2" Jul 24 14:20:38.803934 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:38.803905 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:20:38.804042 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:38.803978 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:20:38.804042 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:38.804029 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:20:39.811415 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:39.811381 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" event={"ID":"b2d16e5a-7726-4c8d-b2a9-e266da469b11","Type":"ContainerStarted","Data":"b415edef06b5093d6e630c05c1afd8f5c459f6d70ef6a1ea4e1b499f1bcaf8d3"} Jul 24 14:20:40.317670 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:40.317638 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:40.317840 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:40.317816 2579 patch_prober.go:28] interesting pod/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 24 14:20:40.317885 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:40.317862 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" podUID="b2d16e5a-7726-4c8d-b2a9-e266da469b11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 24 14:20:41.317719 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:41.317681 2579 patch_prober.go:28] interesting pod/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 24 14:20:41.318156 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:41.317741 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" podUID="b2d16e5a-7726-4c8d-b2a9-e266da469b11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 24 14:20:42.318208 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.318160 2579 patch_prober.go:28] interesting pod/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 24 14:20:42.318614 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.318237 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" podUID="b2d16e5a-7726-4c8d-b2a9-e266da469b11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 24 14:20:42.383538 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.383505 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:42.734918 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.734839 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff"] Jul 24 14:20:42.737557 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.737537 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.741682 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.741654 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-359bb9-openshift-default-dockercfg-fr8t4\"" Jul 24 14:20:42.760019 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.759890 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff"] Jul 24 14:20:42.791682 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.791643 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:42.809919 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.809884 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-workload-certs\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.809919 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.809918 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6hxn\" (UniqueName: \"kubernetes.io/projected/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-kube-api-access-x6hxn\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.810178 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.809939 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-credential-socket\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.810178 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.809959 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-data\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.810178 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.810123 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-token\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.810178 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.810152 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-podinfo\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.810178 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.810173 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-workload-socket\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.810363 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.810215 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-envoy\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.810363 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.810235 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istiod-ca-cert\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.910856 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.910813 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istiod-ca-cert\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.911079 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.910889 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-workload-certs\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.911079 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.910919 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6hxn\" (UniqueName: \"kubernetes.io/projected/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-kube-api-access-x6hxn\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.911079 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.910946 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-credential-socket\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.911079 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.910974 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-data\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.911319 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.911125 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-token\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.911319 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.911158 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-podinfo\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.911319 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.911187 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-workload-socket\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.911319 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.911224 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-envoy\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.911507 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.911388 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-workload-certs\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.911507 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.911445 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-credential-socket\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.911507 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.911496 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-data\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.911686 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.911667 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istiod-ca-cert\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.911725 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.911692 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-workload-socket\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.913598 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.913577 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-envoy\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.913805 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.913786 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-podinfo\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.924009 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.923972 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-token\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:42.924275 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:42.924257 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6hxn\" (UniqueName: \"kubernetes.io/projected/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-kube-api-access-x6hxn\") pod \"e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:43.061173 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:43.060890 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:43.250018 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:43.249257 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff"] Jul 24 14:20:43.270318 ip-10-0-137-60 kubenswrapper[2579]: W0724 14:20:43.270107 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda03fdaeb_1c74_4b7c_94b4_b10f645bc36e.slice/crio-c73992b3630be267d7f747099c557c3a5ffdd4700a9d524929abe5ae955a444a WatchSource:0}: Error finding container c73992b3630be267d7f747099c557c3a5ffdd4700a9d524929abe5ae955a444a: Status 404 returned error can't find the container with id c73992b3630be267d7f747099c557c3a5ffdd4700a9d524929abe5ae955a444a Jul 24 14:20:43.272718 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:43.272685 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:20:43.272842 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:43.272764 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:20:43.272842 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:43.272821 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:20:43.317805 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:43.317737 2579 patch_prober.go:28] interesting pod/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 24 14:20:43.317805 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:43.317791 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" podUID="b2d16e5a-7726-4c8d-b2a9-e266da469b11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 24 14:20:43.603606 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:43.603521 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:43.834729 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:43.834694 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" event={"ID":"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e","Type":"ContainerStarted","Data":"648994023aa90dc79cc63fb23b0da44bc444813025c81d19c40af106522b9e76"} Jul 24 14:20:43.834899 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:43.834753 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" event={"ID":"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e","Type":"ContainerStarted","Data":"c73992b3630be267d7f747099c557c3a5ffdd4700a9d524929abe5ae955a444a"} Jul 24 14:20:43.864147 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:43.864044 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" podStartSLOduration=1.864027322 podStartE2EDuration="1.864027322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:20:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:20:43.861263977 +0000 UTC m=+2785.984646375" watchObservedRunningTime="2026-07-24 14:20:43.864027322 +0000 UTC m=+2785.987409719" Jul 24 14:20:44.061774 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:44.061735 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:44.067956 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:44.067924 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:44.321180 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:44.321150 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:44.321591 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:44.321569 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:44.322330 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:44.322306 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:20:44.840778 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:44.840747 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:44.841658 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:44.841638 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:20:45.882281 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:45.882244 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:46.044332 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:46.044291 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:48.787723 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:48.787694 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:48.968886 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:48.968855 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:49.547150 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:49.547115 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:49.669211 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:49.669179 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:50.432164 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:50.432132 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:55.417024 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:20:55.416958 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:21:03.958055 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:03.958008 2579 generic.go:356] "Generic (PLEG): container finished" podID="a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" containerID="648994023aa90dc79cc63fb23b0da44bc444813025c81d19c40af106522b9e76" exitCode=0 Jul 24 14:21:03.958655 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:03.958068 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" event={"ID":"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e","Type":"ContainerDied","Data":"648994023aa90dc79cc63fb23b0da44bc444813025c81d19c40af106522b9e76"} Jul 24 14:21:03.958655 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:03.958644 2579 scope.go:117] "RemoveContainer" containerID="648994023aa90dc79cc63fb23b0da44bc444813025c81d19c40af106522b9e76" Jul 24 14:21:03.959852 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:03.959813 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:21:03.959956 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:03.959878 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:21:03.959956 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:03.959909 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:21:04.970084 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:04.970038 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" event={"ID":"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e","Type":"ContainerStarted","Data":"5808b76796cb119f25020633a125da5380e85ff929d404d67a819dcc9c5278b0"} Jul 24 14:21:05.062445 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:05.062398 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:21:05.062651 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:05.062533 2579 patch_prober.go:28] interesting pod/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 24 14:21:05.062651 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:05.062576 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" podUID="a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 24 14:21:06.062197 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:06.062153 2579 patch_prober.go:28] interesting pod/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 24 14:21:06.063252 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:06.063211 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" podUID="a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 24 14:21:07.062339 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:07.062293 2579 patch_prober.go:28] interesting pod/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 24 14:21:07.062844 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:07.062365 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" podUID="a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 24 14:21:08.062254 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:08.062207 2579 patch_prober.go:28] interesting pod/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 24 14:21:08.062473 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:08.062293 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" podUID="a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 24 14:21:09.066515 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:09.066482 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:21:09.066884 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:09.066841 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:21:09.067597 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:09.067578 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:21:22.470255 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:22.470217 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:26.461443 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:26.461409 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:21:37.018271 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:37.018225 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:21:53.155515 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.155466 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:53.485079 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.484984 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg"] Jul 24 14:21:53.487832 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.487814 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.491317 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.491293 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-269297-openshift-default-dockercfg-ttkp8\"" Jul 24 14:21:53.509431 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.509403 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:53.511482 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.511454 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg"] Jul 24 14:21:53.554790 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.554283 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-workload-socket\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.554790 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.554341 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b960cd8c-bff2-425a-b07b-914eb382c663-istiod-ca-cert\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.554790 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.554397 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-istio-data\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.554790 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.554479 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-workload-certs\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.554790 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.554513 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72x2q\" (UniqueName: \"kubernetes.io/projected/b960cd8c-bff2-425a-b07b-914eb382c663-kube-api-access-72x2q\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.554790 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.554588 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-credential-socket\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.554790 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.554615 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b960cd8c-bff2-425a-b07b-914eb382c663-istio-podinfo\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.554790 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.554638 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b960cd8c-bff2-425a-b07b-914eb382c663-istio-token\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.554790 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.554697 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-istio-envoy\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.636258 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.636195 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:53.655450 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.655414 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-istio-envoy\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.655450 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.655455 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-workload-socket\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.655684 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.655477 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b960cd8c-bff2-425a-b07b-914eb382c663-istiod-ca-cert\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.655684 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.655511 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-istio-data\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.655684 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.655562 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-workload-certs\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.655684 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.655581 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72x2q\" (UniqueName: \"kubernetes.io/projected/b960cd8c-bff2-425a-b07b-914eb382c663-kube-api-access-72x2q\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.655684 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.655628 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-credential-socket\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.655684 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.655652 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b960cd8c-bff2-425a-b07b-914eb382c663-istio-podinfo\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.655684 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.655676 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b960cd8c-bff2-425a-b07b-914eb382c663-istio-token\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.656050 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.655879 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-workload-socket\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.656050 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.655953 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-istio-data\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.656167 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.656084 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-workload-certs\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.656340 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.656320 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-credential-socket\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.656555 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.656495 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b960cd8c-bff2-425a-b07b-914eb382c663-istiod-ca-cert\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.658077 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.658056 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-istio-envoy\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.658321 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.658303 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b960cd8c-bff2-425a-b07b-914eb382c663-istio-podinfo\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.665301 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.665273 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b960cd8c-bff2-425a-b07b-914eb382c663-istio-token\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.666226 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.666202 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72x2q\" (UniqueName: \"kubernetes.io/projected/b960cd8c-bff2-425a-b07b-914eb382c663-kube-api-access-72x2q\") pod \"e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.798857 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.798760 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:53.954209 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.954174 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg"] Jul 24 14:21:53.955623 ip-10-0-137-60 kubenswrapper[2579]: W0724 14:21:53.955591 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb960cd8c_bff2_425a_b07b_914eb382c663.slice/crio-e51a0b97db2c3ca006c8d59b0ae473f8542b7f800c7b7e8b5f4fe58933f34c79 WatchSource:0}: Error finding container e51a0b97db2c3ca006c8d59b0ae473f8542b7f800c7b7e8b5f4fe58933f34c79: Status 404 returned error can't find the container with id e51a0b97db2c3ca006c8d59b0ae473f8542b7f800c7b7e8b5f4fe58933f34c79 Jul 24 14:21:53.957914 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.957882 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:21:53.957973 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.957949 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:21:53.958040 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:53.957977 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:21:54.262769 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:54.262736 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" event={"ID":"b960cd8c-bff2-425a-b07b-914eb382c663","Type":"ContainerStarted","Data":"4dba882b7fe099d1cc57d8c470b92de4db4501d2a282b16a76d35aa00ee96d5b"} Jul 24 14:21:54.263206 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:54.262791 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" event={"ID":"b960cd8c-bff2-425a-b07b-914eb382c663","Type":"ContainerStarted","Data":"e51a0b97db2c3ca006c8d59b0ae473f8542b7f800c7b7e8b5f4fe58933f34c79"} Jul 24 14:21:54.294600 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:54.294545 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" podStartSLOduration=1.294527528 podStartE2EDuration="1.294527528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:21:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:21:54.289518205 +0000 UTC m=+2856.412900604" watchObservedRunningTime="2026-07-24 14:21:54.294527528 +0000 UTC m=+2856.417909926" Jul 24 14:21:54.799523 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:54.799485 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:55.805197 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:55.805166 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:56.276916 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:56.276829 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:56.277822 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:56.277797 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:21:56.375685 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:56.375641 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:56.471250 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:56.470946 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:59.427199 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:59.427164 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:59.560115 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:21:59.560086 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:22:00.023681 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:00.023640 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:22:00.128242 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:00.128211 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:22:00.923865 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:00.923823 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:22:09.610781 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:09.610687 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:22:18.415953 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:18.415915 2579 generic.go:356] "Generic (PLEG): container finished" podID="b960cd8c-bff2-425a-b07b-914eb382c663" containerID="4dba882b7fe099d1cc57d8c470b92de4db4501d2a282b16a76d35aa00ee96d5b" exitCode=0 Jul 24 14:22:18.421058 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:18.421030 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" event={"ID":"b960cd8c-bff2-425a-b07b-914eb382c663","Type":"ContainerDied","Data":"4dba882b7fe099d1cc57d8c470b92de4db4501d2a282b16a76d35aa00ee96d5b"} Jul 24 14:22:18.421361 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:18.421346 2579 scope.go:117] "RemoveContainer" containerID="4dba882b7fe099d1cc57d8c470b92de4db4501d2a282b16a76d35aa00ee96d5b" Jul 24 14:22:18.422429 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:18.422401 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:22:18.422558 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:18.422468 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:22:18.422558 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:18.422504 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 14:22:19.423711 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:19.423671 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" event={"ID":"b960cd8c-bff2-425a-b07b-914eb382c663","Type":"ContainerStarted","Data":"ed687f7629a7f0ae7b2ae981ceb914fa213a5f56ec638618ccc119598e77341c"} Jul 24 14:22:19.800030 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:19.799899 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:22:19.800206 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:19.800152 2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 24 14:22:19.800264 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:19.800207 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" podUID="b960cd8c-bff2-425a-b07b-914eb382c663" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 24 14:22:20.799172 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:20.799142 2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 24 14:22:20.799589 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:20.799196 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" podUID="b960cd8c-bff2-425a-b07b-914eb382c663" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 24 14:22:21.799312 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:21.799275 2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 24 14:22:21.799701 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:21.799335 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" podUID="b960cd8c-bff2-425a-b07b-914eb382c663" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 24 14:22:22.799607 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:22.799564 2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 24 14:22:22.800028 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:22.799629 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" podUID="b960cd8c-bff2-425a-b07b-914eb382c663" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 24 14:22:23.803235 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:23.803200 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:22:23.803634 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:23.803526 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:22:23.804323 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:23.804308 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:22:30.987427 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:30.987391 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:22:32.443885 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:32.443841 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:22:32.603390 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:32.603354 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:22:37.874959 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:37.874919 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:22:39.746667 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:39.746630 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:22:39.863165 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:39.863128 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:22:40.018448 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:40.018341 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg"] Jul 24 14:22:40.018609 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:40.018575 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" podUID="b960cd8c-bff2-425a-b07b-914eb382c663" containerName="istio-proxy" containerID="cri-o://ed687f7629a7f0ae7b2ae981ceb914fa213a5f56ec638618ccc119598e77341c" gracePeriod=30 Jul 24 14:22:40.050391 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:40.050312 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:22:40.297687 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:40.297556 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:22:45.277211 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.277187 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:22:45.413121 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.413082 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-credential-socket\") pod \"b960cd8c-bff2-425a-b07b-914eb382c663\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " Jul 24 14:22:45.413309 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.413128 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72x2q\" (UniqueName: \"kubernetes.io/projected/b960cd8c-bff2-425a-b07b-914eb382c663-kube-api-access-72x2q\") pod \"b960cd8c-bff2-425a-b07b-914eb382c663\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " Jul 24 14:22:45.413309 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.413184 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b960cd8c-bff2-425a-b07b-914eb382c663-istio-token\") pod \"b960cd8c-bff2-425a-b07b-914eb382c663\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " Jul 24 14:22:45.413309 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.413214 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-istio-data\") pod \"b960cd8c-bff2-425a-b07b-914eb382c663\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " Jul 24 14:22:45.413309 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.413254 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-workload-socket\") pod \"b960cd8c-bff2-425a-b07b-914eb382c663\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " Jul 24 14:22:45.413309 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.413293 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b960cd8c-bff2-425a-b07b-914eb382c663-istio-podinfo\") pod \"b960cd8c-bff2-425a-b07b-914eb382c663\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " Jul 24 14:22:45.413568 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.413344 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b960cd8c-bff2-425a-b07b-914eb382c663-istiod-ca-cert\") pod \"b960cd8c-bff2-425a-b07b-914eb382c663\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " Jul 24 14:22:45.413568 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.413371 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-workload-certs\") pod \"b960cd8c-bff2-425a-b07b-914eb382c663\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " Jul 24 14:22:45.413568 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.413419 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b960cd8c-bff2-425a-b07b-914eb382c663" (UID: "b960cd8c-bff2-425a-b07b-914eb382c663"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:22:45.413568 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.413465 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-istio-envoy\") pod \"b960cd8c-bff2-425a-b07b-914eb382c663\" (UID: \"b960cd8c-bff2-425a-b07b-914eb382c663\") " Jul 24 14:22:45.413770 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.413585 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b960cd8c-bff2-425a-b07b-914eb382c663" (UID: "b960cd8c-bff2-425a-b07b-914eb382c663"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:22:45.414107 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.413879 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b960cd8c-bff2-425a-b07b-914eb382c663-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b960cd8c-bff2-425a-b07b-914eb382c663" (UID: "b960cd8c-bff2-425a-b07b-914eb382c663"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:22:45.414107 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.413952 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-credential-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:22:45.414107 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.414047 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-workload-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:22:45.414301 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.414080 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b960cd8c-bff2-425a-b07b-914eb382c663" (UID: "b960cd8c-bff2-425a-b07b-914eb382c663"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:22:45.414301 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.414188 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-istio-data" (OuterVolumeSpecName: "istio-data") pod "b960cd8c-bff2-425a-b07b-914eb382c663" (UID: "b960cd8c-bff2-425a-b07b-914eb382c663"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:22:45.415629 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.415597 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b960cd8c-bff2-425a-b07b-914eb382c663-istio-token" (OuterVolumeSpecName: "istio-token") pod "b960cd8c-bff2-425a-b07b-914eb382c663" (UID: "b960cd8c-bff2-425a-b07b-914eb382c663"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:22:45.415816 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.415773 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b960cd8c-bff2-425a-b07b-914eb382c663-kube-api-access-72x2q" (OuterVolumeSpecName: "kube-api-access-72x2q") pod "b960cd8c-bff2-425a-b07b-914eb382c663" (UID: "b960cd8c-bff2-425a-b07b-914eb382c663"). InnerVolumeSpecName "kube-api-access-72x2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:22:45.415816 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.415783 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b960cd8c-bff2-425a-b07b-914eb382c663-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b960cd8c-bff2-425a-b07b-914eb382c663" (UID: "b960cd8c-bff2-425a-b07b-914eb382c663"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:22:45.416119 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.416104 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b960cd8c-bff2-425a-b07b-914eb382c663" (UID: "b960cd8c-bff2-425a-b07b-914eb382c663"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:22:45.515414 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.515373 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-istio-data\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:22:45.515414 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.515405 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b960cd8c-bff2-425a-b07b-914eb382c663-istio-podinfo\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:22:45.515414 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.515417 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b960cd8c-bff2-425a-b07b-914eb382c663-istiod-ca-cert\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:22:45.515672 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.515428 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-workload-certs\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:22:45.515672 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.515439 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b960cd8c-bff2-425a-b07b-914eb382c663-istio-envoy\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:22:45.515672 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.515450 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72x2q\" (UniqueName: \"kubernetes.io/projected/b960cd8c-bff2-425a-b07b-914eb382c663-kube-api-access-72x2q\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:22:45.515672 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.515466 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b960cd8c-bff2-425a-b07b-914eb382c663-istio-token\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:22:45.583054 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.583020 2579 generic.go:356] "Generic (PLEG): container finished" podID="b960cd8c-bff2-425a-b07b-914eb382c663" containerID="ed687f7629a7f0ae7b2ae981ceb914fa213a5f56ec638618ccc119598e77341c" exitCode=0 Jul 24 14:22:45.583260 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.583123 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" event={"ID":"b960cd8c-bff2-425a-b07b-914eb382c663","Type":"ContainerDied","Data":"ed687f7629a7f0ae7b2ae981ceb914fa213a5f56ec638618ccc119598e77341c"} Jul 24 14:22:45.583260 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.583134 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" Jul 24 14:22:45.583260 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.583166 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg" event={"ID":"b960cd8c-bff2-425a-b07b-914eb382c663","Type":"ContainerDied","Data":"e51a0b97db2c3ca006c8d59b0ae473f8542b7f800c7b7e8b5f4fe58933f34c79"} Jul 24 14:22:45.583260 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.583190 2579 scope.go:117] "RemoveContainer" containerID="ed687f7629a7f0ae7b2ae981ceb914fa213a5f56ec638618ccc119598e77341c" Jul 24 14:22:45.593938 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.593916 2579 scope.go:117] "RemoveContainer" containerID="4dba882b7fe099d1cc57d8c470b92de4db4501d2a282b16a76d35aa00ee96d5b" Jul 24 14:22:45.602398 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.602381 2579 scope.go:117] "RemoveContainer" containerID="ed687f7629a7f0ae7b2ae981ceb914fa213a5f56ec638618ccc119598e77341c" Jul 24 14:22:45.602638 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:22:45.602618 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed687f7629a7f0ae7b2ae981ceb914fa213a5f56ec638618ccc119598e77341c\": container with ID starting with ed687f7629a7f0ae7b2ae981ceb914fa213a5f56ec638618ccc119598e77341c not found: ID does not exist" containerID="ed687f7629a7f0ae7b2ae981ceb914fa213a5f56ec638618ccc119598e77341c" Jul 24 14:22:45.602712 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.602651 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed687f7629a7f0ae7b2ae981ceb914fa213a5f56ec638618ccc119598e77341c"} err="failed to get container status \"ed687f7629a7f0ae7b2ae981ceb914fa213a5f56ec638618ccc119598e77341c\": rpc error: code = NotFound desc = could not find container \"ed687f7629a7f0ae7b2ae981ceb914fa213a5f56ec638618ccc119598e77341c\": container with ID starting with ed687f7629a7f0ae7b2ae981ceb914fa213a5f56ec638618ccc119598e77341c not found: ID does not exist" Jul 24 14:22:45.602712 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.602675 2579 scope.go:117] "RemoveContainer" containerID="4dba882b7fe099d1cc57d8c470b92de4db4501d2a282b16a76d35aa00ee96d5b" Jul 24 14:22:45.602922 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:22:45.602903 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dba882b7fe099d1cc57d8c470b92de4db4501d2a282b16a76d35aa00ee96d5b\": container with ID starting with 4dba882b7fe099d1cc57d8c470b92de4db4501d2a282b16a76d35aa00ee96d5b not found: ID does not exist" containerID="4dba882b7fe099d1cc57d8c470b92de4db4501d2a282b16a76d35aa00ee96d5b" Jul 24 14:22:45.602965 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.602929 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dba882b7fe099d1cc57d8c470b92de4db4501d2a282b16a76d35aa00ee96d5b"} err="failed to get container status \"4dba882b7fe099d1cc57d8c470b92de4db4501d2a282b16a76d35aa00ee96d5b\": rpc error: code = NotFound desc = could not find container \"4dba882b7fe099d1cc57d8c470b92de4db4501d2a282b16a76d35aa00ee96d5b\": container with ID starting with 4dba882b7fe099d1cc57d8c470b92de4db4501d2a282b16a76d35aa00ee96d5b not found: ID does not exist" Jul 24 14:22:45.608897 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.608873 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg"] Jul 24 14:22:45.612999 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:45.612970 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-269297-openshift-default-f7d9c87c4-x95tg"] Jul 24 14:22:46.420645 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:46.420607 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b960cd8c-bff2-425a-b07b-914eb382c663" path="/var/lib/kubelet/pods/b960cd8c-bff2-425a-b07b-914eb382c663/volumes" Jul 24 14:22:59.068135 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:59.068092 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:22:59.170878 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:59.170845 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:22:59.878210 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:59.878177 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:22:59.980259 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:22:59.980224 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:00.735087 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:23:00.735055 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:23:04.399851 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:23:04.399817 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:04.499833 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:23:04.499797 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:04.600858 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:23:04.600805 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:09.133843 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:23:09.133802 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:09.540095 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:23:09.540007 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:14.620776 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:23:14.620737 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:15.509226 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:23:15.509188 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:23:15.827092 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:23:15.827056 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:52.614745 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:23:52.614658 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:53.287740 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:23:53.287705 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:59.595400 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:23:59.595366 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:59.709204 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:23:59.709159 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:59.883608 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:23:59.883529 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx"] Jul 24 14:23:59.883772 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:23:59.883755 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" podUID="b2d16e5a-7726-4c8d-b2a9-e266da469b11" containerName="istio-proxy" containerID="cri-o://b415edef06b5093d6e630c05c1afd8f5c459f6d70ef6a1ea4e1b499f1bcaf8d3" gracePeriod=30 Jul 24 14:23:59.923575 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:23:59.923539 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:00.158372 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:00.158259 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:05.048102 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.047632 2579 generic.go:356] "Generic (PLEG): container finished" podID="b2d16e5a-7726-4c8d-b2a9-e266da469b11" containerID="b415edef06b5093d6e630c05c1afd8f5c459f6d70ef6a1ea4e1b499f1bcaf8d3" exitCode=0 Jul 24 14:24:05.048102 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.047829 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" event={"ID":"b2d16e5a-7726-4c8d-b2a9-e266da469b11","Type":"ContainerDied","Data":"b415edef06b5093d6e630c05c1afd8f5c459f6d70ef6a1ea4e1b499f1bcaf8d3"} Jul 24 14:24:05.048102 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.047935 2579 scope.go:117] "RemoveContainer" containerID="aa9dfa5e637f602222a8a919f89ef8140bc3b55ae28298c661eaf832404e88d2" Jul 24 14:24:05.144187 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.144130 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:24:05.233202 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.233168 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-data\") pod \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " Jul 24 14:24:05.233372 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.233212 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-workload-socket\") pod \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " Jul 24 14:24:05.233372 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.233233 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istiod-ca-cert\") pod \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " Jul 24 14:24:05.233372 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.233254 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-workload-certs\") pod \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " Jul 24 14:24:05.233372 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.233287 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t49ps\" (UniqueName: \"kubernetes.io/projected/b2d16e5a-7726-4c8d-b2a9-e266da469b11-kube-api-access-t49ps\") pod \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " Jul 24 14:24:05.233573 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.233393 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-podinfo\") pod \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " Jul 24 14:24:05.233573 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.233439 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-envoy\") pod \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " Jul 24 14:24:05.233573 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.233468 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-credential-socket\") pod \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " Jul 24 14:24:05.233573 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.233520 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b2d16e5a-7726-4c8d-b2a9-e266da469b11" (UID: "b2d16e5a-7726-4c8d-b2a9-e266da469b11"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:05.233573 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.233538 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-token\") pod \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\" (UID: \"b2d16e5a-7726-4c8d-b2a9-e266da469b11\") " Jul 24 14:24:05.233851 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.233641 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b2d16e5a-7726-4c8d-b2a9-e266da469b11" (UID: "b2d16e5a-7726-4c8d-b2a9-e266da469b11"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:24:05.233851 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.233753 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b2d16e5a-7726-4c8d-b2a9-e266da469b11" (UID: "b2d16e5a-7726-4c8d-b2a9-e266da469b11"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:05.233981 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.233947 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b2d16e5a-7726-4c8d-b2a9-e266da469b11" (UID: "b2d16e5a-7726-4c8d-b2a9-e266da469b11"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:05.234068 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.234051 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-credential-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:05.234135 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.234072 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-workload-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:05.234135 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.234087 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istiod-ca-cert\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:05.234135 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.234099 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-data" (OuterVolumeSpecName: "istio-data") pod "b2d16e5a-7726-4c8d-b2a9-e266da469b11" (UID: "b2d16e5a-7726-4c8d-b2a9-e266da469b11"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:05.235785 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.235752 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b2d16e5a-7726-4c8d-b2a9-e266da469b11" (UID: "b2d16e5a-7726-4c8d-b2a9-e266da469b11"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:05.235902 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.235795 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b2d16e5a-7726-4c8d-b2a9-e266da469b11" (UID: "b2d16e5a-7726-4c8d-b2a9-e266da469b11"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:24:05.236244 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.236222 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-token" (OuterVolumeSpecName: "istio-token") pod "b2d16e5a-7726-4c8d-b2a9-e266da469b11" (UID: "b2d16e5a-7726-4c8d-b2a9-e266da469b11"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:24:05.236244 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.236229 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d16e5a-7726-4c8d-b2a9-e266da469b11-kube-api-access-t49ps" (OuterVolumeSpecName: "kube-api-access-t49ps") pod "b2d16e5a-7726-4c8d-b2a9-e266da469b11" (UID: "b2d16e5a-7726-4c8d-b2a9-e266da469b11"). InnerVolumeSpecName "kube-api-access-t49ps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:24:05.335017 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.334957 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-workload-certs\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:05.335017 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.335011 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t49ps\" (UniqueName: \"kubernetes.io/projected/b2d16e5a-7726-4c8d-b2a9-e266da469b11-kube-api-access-t49ps\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:05.335017 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.335022 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-podinfo\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:05.335251 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.335031 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-envoy\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:05.335251 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.335038 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-token\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:05.335251 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:05.335046 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d16e5a-7726-4c8d-b2a9-e266da469b11-istio-data\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:06.054324 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:06.054296 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" Jul 24 14:24:06.054324 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:06.054323 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx" event={"ID":"b2d16e5a-7726-4c8d-b2a9-e266da469b11","Type":"ContainerDied","Data":"7ad0da9bc5239bfeba4696c19512e91031eb7a88393c44def70474fe41a7fdea"} Jul 24 14:24:06.054824 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:06.054379 2579 scope.go:117] "RemoveContainer" containerID="b415edef06b5093d6e630c05c1afd8f5c459f6d70ef6a1ea4e1b499f1bcaf8d3" Jul 24 14:24:06.084637 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:06.084602 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx"] Jul 24 14:24:06.091816 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:06.091789 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-546fcb-openshift-default-89bf5f698-tpcxx"] Jul 24 14:24:06.196315 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:06.196283 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:06.420583 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:06.420540 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2d16e5a-7726-4c8d-b2a9-e266da469b11" path="/var/lib/kubelet/pods/b2d16e5a-7726-4c8d-b2a9-e266da469b11/volumes" Jul 24 14:24:11.732720 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:11.732681 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:11.829324 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:11.829291 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:11.930506 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:11.930474 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:14.919634 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:14.919601 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:24:15.579598 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:15.579550 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:15.998311 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:15.998220 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:22.397258 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:22.397215 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:22.498598 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:22.498562 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:22.673061 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:22.672955 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff"] Jul 24 14:24:22.673250 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:22.673214 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" podUID="a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" containerName="istio-proxy" containerID="cri-o://5808b76796cb119f25020633a125da5380e85ff929d404d67a819dcc9c5278b0" gracePeriod=30 Jul 24 14:24:22.689005 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:22.688968 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:22.925744 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:22.925656 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:24.069085 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:24.069050 2579 patch_prober.go:28] interesting pod/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 14:24:24.069479 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:24.069119 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" podUID="a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 14:24:27.930234 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.930210 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:24:27.968571 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.968479 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istiod-ca-cert\") pod \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " Jul 24 14:24:27.968729 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.968601 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-data\") pod \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " Jul 24 14:24:27.968729 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.968654 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-podinfo\") pod \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " Jul 24 14:24:27.968729 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.968680 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-envoy\") pod \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " Jul 24 14:24:27.968898 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.968731 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-workload-certs\") pod \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " Jul 24 14:24:27.968898 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.968790 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-credential-socket\") pod \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " Jul 24 14:24:27.968898 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.968824 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-workload-socket\") pod \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " Jul 24 14:24:27.968898 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.968849 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-token\") pod \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " Jul 24 14:24:27.968898 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.968880 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6hxn\" (UniqueName: \"kubernetes.io/projected/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-kube-api-access-x6hxn\") pod \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\" (UID: \"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e\") " Jul 24 14:24:27.968898 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.968877 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" (UID: "a03fdaeb-1c74-4b7c-94b4-b10f645bc36e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:24:27.969294 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.969269 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" (UID: "a03fdaeb-1c74-4b7c-94b4-b10f645bc36e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:27.969483 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.969273 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istiod-ca-cert\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:27.969758 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.969465 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" (UID: "a03fdaeb-1c74-4b7c-94b4-b10f645bc36e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:27.969860 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.969531 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-data" (OuterVolumeSpecName: "istio-data") pod "a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" (UID: "a03fdaeb-1c74-4b7c-94b4-b10f645bc36e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:27.969860 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.969578 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" (UID: "a03fdaeb-1c74-4b7c-94b4-b10f645bc36e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:27.971221 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.971198 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-kube-api-access-x6hxn" (OuterVolumeSpecName: "kube-api-access-x6hxn") pod "a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" (UID: "a03fdaeb-1c74-4b7c-94b4-b10f645bc36e"). InnerVolumeSpecName "kube-api-access-x6hxn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:24:27.971683 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.971662 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-token" (OuterVolumeSpecName: "istio-token") pod "a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" (UID: "a03fdaeb-1c74-4b7c-94b4-b10f645bc36e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:24:27.971763 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.971727 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" (UID: "a03fdaeb-1c74-4b7c-94b4-b10f645bc36e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:27.971869 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:27.971850 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" (UID: "a03fdaeb-1c74-4b7c-94b4-b10f645bc36e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:24:28.071136 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.071101 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-data\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:28.071136 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.071132 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-podinfo\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:28.071136 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.071141 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-envoy\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:28.071375 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.071150 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-workload-certs\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:28.071375 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.071158 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-credential-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:28.071375 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.071166 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-workload-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:28.071375 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.071174 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-istio-token\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:28.071375 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.071182 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6hxn\" (UniqueName: \"kubernetes.io/projected/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e-kube-api-access-x6hxn\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:28.181767 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.181728 2579 generic.go:356] "Generic (PLEG): container finished" podID="a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" containerID="5808b76796cb119f25020633a125da5380e85ff929d404d67a819dcc9c5278b0" exitCode=0 Jul 24 14:24:28.181935 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.181772 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" event={"ID":"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e","Type":"ContainerDied","Data":"5808b76796cb119f25020633a125da5380e85ff929d404d67a819dcc9c5278b0"} Jul 24 14:24:28.181935 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.181789 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" Jul 24 14:24:28.181935 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.181848 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff" event={"ID":"a03fdaeb-1c74-4b7c-94b4-b10f645bc36e","Type":"ContainerDied","Data":"c73992b3630be267d7f747099c557c3a5ffdd4700a9d524929abe5ae955a444a"} Jul 24 14:24:28.181935 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.181870 2579 scope.go:117] "RemoveContainer" containerID="5808b76796cb119f25020633a125da5380e85ff929d404d67a819dcc9c5278b0" Jul 24 14:24:28.193864 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.193844 2579 scope.go:117] "RemoveContainer" containerID="648994023aa90dc79cc63fb23b0da44bc444813025c81d19c40af106522b9e76" Jul 24 14:24:28.202419 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.202401 2579 scope.go:117] "RemoveContainer" containerID="5808b76796cb119f25020633a125da5380e85ff929d404d67a819dcc9c5278b0" Jul 24 14:24:28.202691 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:24:28.202670 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5808b76796cb119f25020633a125da5380e85ff929d404d67a819dcc9c5278b0\": container with ID starting with 5808b76796cb119f25020633a125da5380e85ff929d404d67a819dcc9c5278b0 not found: ID does not exist" containerID="5808b76796cb119f25020633a125da5380e85ff929d404d67a819dcc9c5278b0" Jul 24 14:24:28.202781 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.202702 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5808b76796cb119f25020633a125da5380e85ff929d404d67a819dcc9c5278b0"} err="failed to get container status \"5808b76796cb119f25020633a125da5380e85ff929d404d67a819dcc9c5278b0\": rpc error: code = NotFound desc = could not find container \"5808b76796cb119f25020633a125da5380e85ff929d404d67a819dcc9c5278b0\": container with ID starting with 5808b76796cb119f25020633a125da5380e85ff929d404d67a819dcc9c5278b0 not found: ID does not exist" Jul 24 14:24:28.202781 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.202728 2579 scope.go:117] "RemoveContainer" containerID="648994023aa90dc79cc63fb23b0da44bc444813025c81d19c40af106522b9e76" Jul 24 14:24:28.203009 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:24:28.202972 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"648994023aa90dc79cc63fb23b0da44bc444813025c81d19c40af106522b9e76\": container with ID starting with 648994023aa90dc79cc63fb23b0da44bc444813025c81d19c40af106522b9e76 not found: ID does not exist" containerID="648994023aa90dc79cc63fb23b0da44bc444813025c81d19c40af106522b9e76" Jul 24 14:24:28.203057 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.203013 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"648994023aa90dc79cc63fb23b0da44bc444813025c81d19c40af106522b9e76"} err="failed to get container status \"648994023aa90dc79cc63fb23b0da44bc444813025c81d19c40af106522b9e76\": rpc error: code = NotFound desc = could not find container \"648994023aa90dc79cc63fb23b0da44bc444813025c81d19c40af106522b9e76\": container with ID starting with 648994023aa90dc79cc63fb23b0da44bc444813025c81d19c40af106522b9e76 not found: ID does not exist" Jul 24 14:24:28.210467 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.210443 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff"] Jul 24 14:24:28.215596 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.215574 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-359bb9-openshift-default-547467945b-hkwff"] Jul 24 14:24:28.420957 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:28.420922 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" path="/var/lib/kubelet/pods/a03fdaeb-1c74-4b7c-94b4-b10f645bc36e/volumes" Jul 24 14:24:34.038843 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:34.038812 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:34.156974 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:34.156930 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:34.255852 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:34.255803 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:37.414005 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:37.413955 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:24:38.177223 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:38.177189 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:38.627676 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:38.627638 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:46.010089 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:46.010030 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:46.113364 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:46.113306 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:46.297275 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:46.297187 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c"] Jul 24 14:24:46.297447 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:46.297407 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" podUID="6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" containerName="istio-proxy" containerID="cri-o://7f78e970a1adc5b6fd211552af665fe1584921b27ae6eb7d4ee9c46e746139cb" gracePeriod=30 Jul 24 14:24:46.329154 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:46.329117 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:46.575348 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:46.575315 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:50.707982 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:50.707946 2579 patch_prober.go:28] interesting pod/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 14:24:50.708433 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:50.708025 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" podUID="6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 14:24:51.554883 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.554858 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:24:51.710062 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.709951 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istiod-ca-cert\") pod \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " Jul 24 14:24:51.710062 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.710035 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-credential-socket\") pod \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " Jul 24 14:24:51.710566 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.710072 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-token\") pod \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " Jul 24 14:24:51.710566 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.710093 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-workload-certs\") pod \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " Jul 24 14:24:51.710566 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.710114 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgdgn\" (UniqueName: \"kubernetes.io/projected/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-kube-api-access-sgdgn\") pod \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " Jul 24 14:24:51.710566 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.710139 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-workload-socket\") pod \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " Jul 24 14:24:51.710566 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.710216 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-data\") pod \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " Jul 24 14:24:51.710566 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.710248 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-podinfo\") pod \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " Jul 24 14:24:51.710566 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.710271 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-envoy\") pod \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\" (UID: \"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8\") " Jul 24 14:24:51.710566 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.710431 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" (UID: "6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:51.710566 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.710446 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" (UID: "6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:51.710566 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.710462 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" (UID: "6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:24:51.711291 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.710686 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istiod-ca-cert\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:51.711291 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.710705 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-credential-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:51.711291 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.710706 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" (UID: "6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:51.711291 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.710722 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-workload-certs\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:51.711291 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.711223 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-data" (OuterVolumeSpecName: "istio-data") pod "6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" (UID: "6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:51.712540 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.712513 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" (UID: "6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:51.712711 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.712686 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" (UID: "6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:24:51.712980 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.712961 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-token" (OuterVolumeSpecName: "istio-token") pod "6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" (UID: "6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:24:51.713053 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.712961 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-kube-api-access-sgdgn" (OuterVolumeSpecName: "kube-api-access-sgdgn") pod "6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" (UID: "6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8"). InnerVolumeSpecName "kube-api-access-sgdgn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:24:51.811846 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.811807 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-data\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:51.811846 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.811839 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-podinfo\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:51.811846 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.811848 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-envoy\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:51.811846 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.811856 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-istio-token\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:51.812136 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.811864 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgdgn\" (UniqueName: \"kubernetes.io/projected/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-kube-api-access-sgdgn\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:51.812136 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:51.811873 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8-workload-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:24:52.329714 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:52.329672 2579 generic.go:356] "Generic (PLEG): container finished" podID="6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" containerID="7f78e970a1adc5b6fd211552af665fe1584921b27ae6eb7d4ee9c46e746139cb" exitCode=0 Jul 24 14:24:52.329900 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:52.329737 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" event={"ID":"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8","Type":"ContainerDied","Data":"7f78e970a1adc5b6fd211552af665fe1584921b27ae6eb7d4ee9c46e746139cb"} Jul 24 14:24:52.329900 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:52.329751 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" Jul 24 14:24:52.329900 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:52.329822 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c" event={"ID":"6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8","Type":"ContainerDied","Data":"d52a348004fe992756b632aa455313fcb481277db39427bf4a3504b7cfd1a5f2"} Jul 24 14:24:52.329900 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:52.329846 2579 scope.go:117] "RemoveContainer" containerID="7f78e970a1adc5b6fd211552af665fe1584921b27ae6eb7d4ee9c46e746139cb" Jul 24 14:24:52.339774 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:52.339754 2579 scope.go:117] "RemoveContainer" containerID="52f77fe2ade8fdc0cdbc76784d6db9c2f86a4c145523c0937b1c0622518d2b82" Jul 24 14:24:52.348297 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:52.348273 2579 scope.go:117] "RemoveContainer" containerID="7f78e970a1adc5b6fd211552af665fe1584921b27ae6eb7d4ee9c46e746139cb" Jul 24 14:24:52.348624 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:24:52.348602 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f78e970a1adc5b6fd211552af665fe1584921b27ae6eb7d4ee9c46e746139cb\": container with ID starting with 7f78e970a1adc5b6fd211552af665fe1584921b27ae6eb7d4ee9c46e746139cb not found: ID does not exist" containerID="7f78e970a1adc5b6fd211552af665fe1584921b27ae6eb7d4ee9c46e746139cb" Jul 24 14:24:52.348681 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:52.348634 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f78e970a1adc5b6fd211552af665fe1584921b27ae6eb7d4ee9c46e746139cb"} err="failed to get container status \"7f78e970a1adc5b6fd211552af665fe1584921b27ae6eb7d4ee9c46e746139cb\": rpc error: code = NotFound desc = could not find container \"7f78e970a1adc5b6fd211552af665fe1584921b27ae6eb7d4ee9c46e746139cb\": container with ID starting with 7f78e970a1adc5b6fd211552af665fe1584921b27ae6eb7d4ee9c46e746139cb not found: ID does not exist" Jul 24 14:24:52.348681 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:52.348652 2579 scope.go:117] "RemoveContainer" containerID="52f77fe2ade8fdc0cdbc76784d6db9c2f86a4c145523c0937b1c0622518d2b82" Jul 24 14:24:52.348907 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:24:52.348887 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52f77fe2ade8fdc0cdbc76784d6db9c2f86a4c145523c0937b1c0622518d2b82\": container with ID starting with 52f77fe2ade8fdc0cdbc76784d6db9c2f86a4c145523c0937b1c0622518d2b82 not found: ID does not exist" containerID="52f77fe2ade8fdc0cdbc76784d6db9c2f86a4c145523c0937b1c0622518d2b82" Jul 24 14:24:52.348964 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:52.348918 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52f77fe2ade8fdc0cdbc76784d6db9c2f86a4c145523c0937b1c0622518d2b82"} err="failed to get container status \"52f77fe2ade8fdc0cdbc76784d6db9c2f86a4c145523c0937b1c0622518d2b82\": rpc error: code = NotFound desc = could not find container \"52f77fe2ade8fdc0cdbc76784d6db9c2f86a4c145523c0937b1c0622518d2b82\": container with ID starting with 52f77fe2ade8fdc0cdbc76784d6db9c2f86a4c145523c0937b1c0622518d2b82 not found: ID does not exist" Jul 24 14:24:52.354517 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:52.354494 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c"] Jul 24 14:24:52.362611 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:52.362589 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-4cd364-openshift-default-59bfc59d8-hqz4c"] Jul 24 14:24:52.421338 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:52.421305 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" path="/var/lib/kubelet/pods/6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8/volumes" Jul 24 14:24:57.671802 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:57.671768 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:57.793603 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:57.793571 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:57.894852 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:57.894809 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:59.353672 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:59.353629 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:59.454403 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:24:59.454369 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:00.718015 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:00.717946 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqc45"] Jul 24 14:25:01.796269 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:01.796230 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:02.223415 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:02.223379 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:08.779361 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:08.779281 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:09.012308 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:09.012277 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:09.119447 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:09.119412 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k"] Jul 24 14:25:09.119638 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:09.119617 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" podUID="1f2f414b-9b83-4d8d-8ef3-4582121f3532" containerName="istio-proxy" containerID="cri-o://ad2b09a6d7e7b14ec9db57dcb0d9b2ddd6d8f2409aa09db89b22d3b767d832ea" gracePeriod=30 Jul 24 14:25:09.150372 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:09.150338 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:09.322092 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:09.322049 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:13.993087 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:13.993047 2579 patch_prober.go:28] interesting pod/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 14:25:13.993456 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:13.993123 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" podUID="1f2f414b-9b83-4d8d-8ef3-4582121f3532" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 14:25:14.372209 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.372178 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:25:14.420392 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.420358 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-workload-socket\") pod \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " Jul 24 14:25:14.420584 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.420397 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-envoy\") pod \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " Jul 24 14:25:14.420584 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.420435 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-podinfo\") pod \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " Jul 24 14:25:14.420584 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.420476 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-workload-certs\") pod \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " Jul 24 14:25:14.420584 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.420530 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7z4zz\" (UniqueName: \"kubernetes.io/projected/1f2f414b-9b83-4d8d-8ef3-4582121f3532-kube-api-access-7z4zz\") pod \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " Jul 24 14:25:14.420584 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.420575 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-token\") pod \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " Jul 24 14:25:14.420861 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.420606 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-credential-socket\") pod \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " Jul 24 14:25:14.420861 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.420645 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-data\") pod \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " Jul 24 14:25:14.420861 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.420677 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istiod-ca-cert\") pod \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\" (UID: \"1f2f414b-9b83-4d8d-8ef3-4582121f3532\") " Jul 24 14:25:14.420861 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.420685 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1f2f414b-9b83-4d8d-8ef3-4582121f3532" (UID: "1f2f414b-9b83-4d8d-8ef3-4582121f3532"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:25:14.421115 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.421096 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-workload-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:25:14.421511 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.421450 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1f2f414b-9b83-4d8d-8ef3-4582121f3532" (UID: "1f2f414b-9b83-4d8d-8ef3-4582121f3532"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:25:14.421670 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.421645 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1f2f414b-9b83-4d8d-8ef3-4582121f3532" (UID: "1f2f414b-9b83-4d8d-8ef3-4582121f3532"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:25:14.421743 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.421312 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1f2f414b-9b83-4d8d-8ef3-4582121f3532" (UID: "1f2f414b-9b83-4d8d-8ef3-4582121f3532"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:25:14.422442 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.422398 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-data" (OuterVolumeSpecName: "istio-data") pod "1f2f414b-9b83-4d8d-8ef3-4582121f3532" (UID: "1f2f414b-9b83-4d8d-8ef3-4582121f3532"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:25:14.423181 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.423153 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1f2f414b-9b83-4d8d-8ef3-4582121f3532" (UID: "1f2f414b-9b83-4d8d-8ef3-4582121f3532"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:25:14.423485 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.423450 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f2f414b-9b83-4d8d-8ef3-4582121f3532-kube-api-access-7z4zz" (OuterVolumeSpecName: "kube-api-access-7z4zz") pod "1f2f414b-9b83-4d8d-8ef3-4582121f3532" (UID: "1f2f414b-9b83-4d8d-8ef3-4582121f3532"). InnerVolumeSpecName "kube-api-access-7z4zz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:25:14.423568 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.423537 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1f2f414b-9b83-4d8d-8ef3-4582121f3532" (UID: "1f2f414b-9b83-4d8d-8ef3-4582121f3532"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:25:14.423568 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.423546 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-token" (OuterVolumeSpecName: "istio-token") pod "1f2f414b-9b83-4d8d-8ef3-4582121f3532" (UID: "1f2f414b-9b83-4d8d-8ef3-4582121f3532"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:25:14.455765 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.455732 2579 generic.go:356] "Generic (PLEG): container finished" podID="1f2f414b-9b83-4d8d-8ef3-4582121f3532" containerID="ad2b09a6d7e7b14ec9db57dcb0d9b2ddd6d8f2409aa09db89b22d3b767d832ea" exitCode=0 Jul 24 14:25:14.455890 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.455779 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" Jul 24 14:25:14.455890 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.455810 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" event={"ID":"1f2f414b-9b83-4d8d-8ef3-4582121f3532","Type":"ContainerDied","Data":"ad2b09a6d7e7b14ec9db57dcb0d9b2ddd6d8f2409aa09db89b22d3b767d832ea"} Jul 24 14:25:14.455890 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.455866 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k" event={"ID":"1f2f414b-9b83-4d8d-8ef3-4582121f3532","Type":"ContainerDied","Data":"b9950aa9f74a7e302ca181163111178e390145eb60bcfb95d60f19d5b530e0fe"} Jul 24 14:25:14.456025 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.455890 2579 scope.go:117] "RemoveContainer" containerID="ad2b09a6d7e7b14ec9db57dcb0d9b2ddd6d8f2409aa09db89b22d3b767d832ea" Jul 24 14:25:14.467022 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.466978 2579 scope.go:117] "RemoveContainer" containerID="d9c5f878bb54c39b56f9730f96b47d491ab68567f34a91051f600bd0d672572a" Jul 24 14:25:14.476130 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.476104 2579 scope.go:117] "RemoveContainer" containerID="ad2b09a6d7e7b14ec9db57dcb0d9b2ddd6d8f2409aa09db89b22d3b767d832ea" Jul 24 14:25:14.476462 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:25:14.476433 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad2b09a6d7e7b14ec9db57dcb0d9b2ddd6d8f2409aa09db89b22d3b767d832ea\": container with ID starting with ad2b09a6d7e7b14ec9db57dcb0d9b2ddd6d8f2409aa09db89b22d3b767d832ea not found: ID does not exist" containerID="ad2b09a6d7e7b14ec9db57dcb0d9b2ddd6d8f2409aa09db89b22d3b767d832ea" Jul 24 14:25:14.476543 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.476472 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad2b09a6d7e7b14ec9db57dcb0d9b2ddd6d8f2409aa09db89b22d3b767d832ea"} err="failed to get container status \"ad2b09a6d7e7b14ec9db57dcb0d9b2ddd6d8f2409aa09db89b22d3b767d832ea\": rpc error: code = NotFound desc = could not find container \"ad2b09a6d7e7b14ec9db57dcb0d9b2ddd6d8f2409aa09db89b22d3b767d832ea\": container with ID starting with ad2b09a6d7e7b14ec9db57dcb0d9b2ddd6d8f2409aa09db89b22d3b767d832ea not found: ID does not exist" Jul 24 14:25:14.476543 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.476496 2579 scope.go:117] "RemoveContainer" containerID="d9c5f878bb54c39b56f9730f96b47d491ab68567f34a91051f600bd0d672572a" Jul 24 14:25:14.477406 ip-10-0-137-60 kubenswrapper[2579]: E0724 14:25:14.477376 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9c5f878bb54c39b56f9730f96b47d491ab68567f34a91051f600bd0d672572a\": container with ID starting with d9c5f878bb54c39b56f9730f96b47d491ab68567f34a91051f600bd0d672572a not found: ID does not exist" containerID="d9c5f878bb54c39b56f9730f96b47d491ab68567f34a91051f600bd0d672572a" Jul 24 14:25:14.477495 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.477412 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c5f878bb54c39b56f9730f96b47d491ab68567f34a91051f600bd0d672572a"} err="failed to get container status \"d9c5f878bb54c39b56f9730f96b47d491ab68567f34a91051f600bd0d672572a\": rpc error: code = NotFound desc = could not find container \"d9c5f878bb54c39b56f9730f96b47d491ab68567f34a91051f600bd0d672572a\": container with ID starting with d9c5f878bb54c39b56f9730f96b47d491ab68567f34a91051f600bd0d672572a not found: ID does not exist" Jul 24 14:25:14.479711 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.479692 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k"] Jul 24 14:25:14.484037 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.484015 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-525009-openshift-default-758db9b4dd-65f4k"] Jul 24 14:25:14.522288 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.522256 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-token\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:25:14.522288 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.522283 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-credential-socket\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:25:14.522288 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.522293 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-data\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:25:14.522486 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.522302 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istiod-ca-cert\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:25:14.522486 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.522310 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-envoy\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:25:14.522486 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.522317 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f2f414b-9b83-4d8d-8ef3-4582121f3532-istio-podinfo\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:25:14.522486 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.522325 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f2f414b-9b83-4d8d-8ef3-4582121f3532-workload-certs\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:25:14.522486 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:14.522333 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z4zz\" (UniqueName: \"kubernetes.io/projected/1f2f414b-9b83-4d8d-8ef3-4582121f3532-kube-api-access-7z4zz\") on node \"ip-10-0-137-60.ec2.internal\" DevicePath \"\"" Jul 24 14:25:15.277351 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:15.277322 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-h49g2_e088b46b-0f4e-4740-a42c-bcd1d04be8c8/manager/0.log" Jul 24 14:25:15.410301 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:15.410272 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4rfzs_4424e0d0-b6e6-44a0-82e9-a098570c212a/manager/0.log" Jul 24 14:25:15.665061 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:15.665031 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hfvdp_bd54ca4e-848b-4c2d-8be0-10214ff2148c/server/0.log" Jul 24 14:25:15.782530 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:15.782494 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-b4ln4_24b46733-ac33-433b-bf13-17bbbbdfc040/manager/0.log" Jul 24 14:25:16.150263 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:16.150227 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85fcbb566-fmjsf_ba47ec72-ddad-41a3-906d-61eae421e7ab/copy-manifests/0.log" Jul 24 14:25:16.163223 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:16.163191 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85fcbb566-fmjsf_ba47ec72-ddad-41a3-906d-61eae421e7ab/manager/0.log" Jul 24 14:25:16.420467 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:16.420383 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f2f414b-9b83-4d8d-8ef3-4582121f3532" path="/var/lib/kubelet/pods/1f2f414b-9b83-4d8d-8ef3-4582121f3532/volumes" Jul 24 14:25:16.646871 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:16.646837 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748345-klnkl_4c7372e4-8852-45d3-9879-ceb95f2f0387/cleanup/0.log" Jul 24 14:25:16.758343 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:16.758236 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748360-ljc7k_7ef16b8e-45e6-48d6-9940-361a9184a0ce/cleanup/0.log" Jul 24 14:25:17.812325 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:17.812298 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm_2a16ceb0-9322-4f1e-b832-434602ebaf83/util/0.log" Jul 24 14:25:17.822094 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:17.822064 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm_2a16ceb0-9322-4f1e-b832-434602ebaf83/pull/0.log" Jul 24 14:25:17.842955 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:17.842911 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm_2a16ceb0-9322-4f1e-b832-434602ebaf83/extract/0.log" Jul 24 14:25:17.963691 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:17.963661 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf_b9f13076-7dba-4f74-8cbd-7be65860bb40/util/0.log" Jul 24 14:25:17.972919 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:17.972892 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf_b9f13076-7dba-4f74-8cbd-7be65860bb40/pull/0.log" Jul 24 14:25:17.979852 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:17.979821 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf_b9f13076-7dba-4f74-8cbd-7be65860bb40/extract/0.log" Jul 24 14:25:18.096474 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:18.096399 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf_10be9861-183d-4216-8429-8a7176d40441/util/0.log" Jul 24 14:25:18.105608 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:18.105583 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf_10be9861-183d-4216-8429-8a7176d40441/pull/0.log" Jul 24 14:25:18.113182 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:18.113161 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf_10be9861-183d-4216-8429-8a7176d40441/extract/0.log" Jul 24 14:25:18.230074 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:18.230046 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf_d3174010-56ea-472b-b6c0-f74cb8bcde98/util/0.log" Jul 24 14:25:18.238494 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:18.238474 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf_d3174010-56ea-472b-b6c0-f74cb8bcde98/pull/0.log" Jul 24 14:25:18.246274 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:18.246249 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf_d3174010-56ea-472b-b6c0-f74cb8bcde98/extract/0.log" Jul 24 14:25:18.990031 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:18.989977 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-r69tn_7d33852d-3cbc-4ab4-9eef-ffaad1841a87/manager/0.log" Jul 24 14:25:19.113444 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:19.113420 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-pqc45_a0e8e60d-dce0-49c3-b42a-a091b9a0eb2a/limitador/0.log" Jul 24 14:25:19.617662 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:19.617623 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw_52323b36-8263-4bff-9712-b2b903cd4204/istio-proxy/0.log" Jul 24 14:25:19.748582 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:19.748552 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kxsk9_d685f64b-4b49-4930-b50a-b45df2488ba2/discovery/0.log" Jul 24 14:25:19.863722 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:19.863695 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85dc78bf9-5v5rs_ef5cbd04-8ca5-4930-82a3-a722b0ab36d3/kube-auth-proxy/0.log" Jul 24 14:25:20.110480 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:20.110451 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-l5w2m_0557c33f-f425-4707-90fc-085902eac820/istio-proxy/1.log" Jul 24 14:25:21.298140 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:21.298107 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-99fz2_1b6538bb-2afe-462a-a845-8e2348f5155f/main/0.log" Jul 24 14:25:26.725668 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:26.725634 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:26.887421 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:26.887388 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:29.545985 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:29.545947 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bxddb_3fdc7954-a53e-440d-b8fa-889eaf069e9a/global-pull-secret-syncer/0.log" Jul 24 14:25:29.695182 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:29.695148 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7sc7n_0c2cbbed-7c36-485b-a62d-bd4abba5d562/konnectivity-agent/0.log" Jul 24 14:25:29.838068 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:29.838038 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-60.ec2.internal_2bb58ddfc8f4df1c84ed6456fbda1a1a/haproxy/0.log" Jul 24 14:25:33.476379 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:33.476353 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm_2a16ceb0-9322-4f1e-b832-434602ebaf83/extract/0.log" Jul 24 14:25:33.503420 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:33.503397 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm_2a16ceb0-9322-4f1e-b832-434602ebaf83/util/0.log" Jul 24 14:25:33.537208 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:33.537182 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q9bkm_2a16ceb0-9322-4f1e-b832-434602ebaf83/pull/0.log" Jul 24 14:25:33.573197 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:33.573166 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf_b9f13076-7dba-4f74-8cbd-7be65860bb40/extract/0.log" Jul 24 14:25:33.600345 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:33.600323 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf_b9f13076-7dba-4f74-8cbd-7be65860bb40/util/0.log" Jul 24 14:25:33.624964 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:33.624937 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0g27xf_b9f13076-7dba-4f74-8cbd-7be65860bb40/pull/0.log" Jul 24 14:25:33.680220 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:33.680193 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf_10be9861-183d-4216-8429-8a7176d40441/extract/0.log" Jul 24 14:25:33.727775 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:33.727679 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf_10be9861-183d-4216-8429-8a7176d40441/util/0.log" Jul 24 14:25:33.758476 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:33.758445 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r77wf_10be9861-183d-4216-8429-8a7176d40441/pull/0.log" Jul 24 14:25:33.794913 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:33.794883 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf_d3174010-56ea-472b-b6c0-f74cb8bcde98/extract/0.log" Jul 24 14:25:33.826878 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:33.826849 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf_d3174010-56ea-472b-b6c0-f74cb8bcde98/util/0.log" Jul 24 14:25:33.860505 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:33.860471 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18x6qf_d3174010-56ea-472b-b6c0-f74cb8bcde98/pull/0.log" Jul 24 14:25:34.675228 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:34.675192 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-r69tn_7d33852d-3cbc-4ab4-9eef-ffaad1841a87/manager/0.log" Jul 24 14:25:34.710953 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:34.710920 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-pqc45_a0e8e60d-dce0-49c3-b42a-a091b9a0eb2a/limitador/0.log" Jul 24 14:25:36.355340 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:36.355316 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d49f3a1a-c07e-4ea0-baac-94778e221398/alertmanager/0.log" Jul 24 14:25:36.380508 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:36.380475 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d49f3a1a-c07e-4ea0-baac-94778e221398/config-reloader/0.log" Jul 24 14:25:36.416554 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:36.416523 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d49f3a1a-c07e-4ea0-baac-94778e221398/kube-rbac-proxy-web/0.log" Jul 24 14:25:36.448766 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:36.448718 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d49f3a1a-c07e-4ea0-baac-94778e221398/kube-rbac-proxy/0.log" Jul 24 14:25:36.476560 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:36.476530 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d49f3a1a-c07e-4ea0-baac-94778e221398/kube-rbac-proxy-metric/0.log" Jul 24 14:25:36.503000 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:36.502968 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d49f3a1a-c07e-4ea0-baac-94778e221398/prom-label-proxy/0.log" Jul 24 14:25:36.530262 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:36.530225 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d49f3a1a-c07e-4ea0-baac-94778e221398/init-config-reloader/0.log" Jul 24 14:25:36.603069 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:36.603033 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-zgvfb_a1fcd557-e028-474d-8114-1c3d52016712/kube-state-metrics/0.log" Jul 24 14:25:36.632169 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:36.632089 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-zgvfb_a1fcd557-e028-474d-8114-1c3d52016712/kube-rbac-proxy-main/0.log" Jul 24 14:25:36.662035 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:36.661980 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-zgvfb_a1fcd557-e028-474d-8114-1c3d52016712/kube-rbac-proxy-self/0.log" Jul 24 14:25:36.692205 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:36.692183 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-698d4d8b68-bzxrc_bbef461c-8313-4bea-8810-8fe7340cb033/metrics-server/0.log" Jul 24 14:25:36.730796 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:36.730769 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-vnbmn_fdd282a6-b290-4a78-9a81-d078d731c942/monitoring-plugin/0.log" Jul 24 14:25:36.884752 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:36.884676 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-db65f_c33fa643-8a39-47d5-bdfc-2ed9529413d4/node-exporter/0.log" Jul 24 14:25:36.910322 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:36.910298 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-db65f_c33fa643-8a39-47d5-bdfc-2ed9529413d4/kube-rbac-proxy/0.log" Jul 24 14:25:36.938005 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:36.937976 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-db65f_c33fa643-8a39-47d5-bdfc-2ed9529413d4/init-textfile/0.log" Jul 24 14:25:37.558008 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:37.557960 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-9mr2c_c5de7f2e-de24-42a5-9f53-6b7ef1f18343/prometheus-operator-admission-webhook/0.log" Jul 24 14:25:37.598323 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:37.598298 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c7b48dcb7-pwhvv_4d254863-fa16-463a-8f54-7777a6df5d8a/telemeter-client/0.log" Jul 24 14:25:37.640739 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:37.640713 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c7b48dcb7-pwhvv_4d254863-fa16-463a-8f54-7777a6df5d8a/reload/0.log" Jul 24 14:25:37.681363 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:37.681336 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c7b48dcb7-pwhvv_4d254863-fa16-463a-8f54-7777a6df5d8a/kube-rbac-proxy/0.log" Jul 24 14:25:39.983107 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983073 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp"] Jul 24 14:25:39.983484 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983437 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f2f414b-9b83-4d8d-8ef3-4582121f3532" containerName="istio-proxy" Jul 24 14:25:39.983484 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983447 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2f414b-9b83-4d8d-8ef3-4582121f3532" containerName="istio-proxy" Jul 24 14:25:39.983484 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983459 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f2f414b-9b83-4d8d-8ef3-4582121f3532" containerName="istio-proxy" Jul 24 14:25:39.983484 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983465 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2f414b-9b83-4d8d-8ef3-4582121f3532" containerName="istio-proxy" Jul 24 14:25:39.983484 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983473 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" containerName="istio-proxy" Jul 24 14:25:39.983484 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983478 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" containerName="istio-proxy" Jul 24 14:25:39.983484 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983486 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d16e5a-7726-4c8d-b2a9-e266da469b11" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983492 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d16e5a-7726-4c8d-b2a9-e266da469b11" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983500 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b960cd8c-bff2-425a-b07b-914eb382c663" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983505 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b960cd8c-bff2-425a-b07b-914eb382c663" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983512 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d16e5a-7726-4c8d-b2a9-e266da469b11" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983518 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d16e5a-7726-4c8d-b2a9-e266da469b11" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983525 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983530 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983536 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983540 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983548 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983553 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983560 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b960cd8c-bff2-425a-b07b-914eb382c663" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983564 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b960cd8c-bff2-425a-b07b-914eb382c663" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983626 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d16e5a-7726-4c8d-b2a9-e266da469b11" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983632 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983640 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983646 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b960cd8c-bff2-425a-b07b-914eb382c663" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983651 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b960cd8c-bff2-425a-b07b-914eb382c663" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983657 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f2f414b-9b83-4d8d-8ef3-4582121f3532" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983662 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ba7df0f-00c7-45d1-a5fa-8dd9af79e6f8" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983667 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d16e5a-7726-4c8d-b2a9-e266da469b11" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983673 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f2f414b-9b83-4d8d-8ef3-4582121f3532" containerName="istio-proxy" Jul 24 14:25:39.983703 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.983678 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a03fdaeb-1c74-4b7c-94b4-b10f645bc36e" containerName="istio-proxy" Jul 24 14:25:39.987113 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.987096 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:39.990689 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.990669 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hh2p5\"/\"kube-root-ca.crt\"" Jul 24 14:25:39.991954 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.991937 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hh2p5\"/\"openshift-service-ca.crt\"" Jul 24 14:25:39.992080 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.991959 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hh2p5\"/\"default-dockercfg-gvc2t\"" Jul 24 14:25:39.998300 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:39.998279 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp"] Jul 24 14:25:40.169170 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.169134 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/365345c0-ad1a-4e97-af0c-89b86769101e-podres\") pod \"perf-node-gather-daemonset-c5cbp\" (UID: \"365345c0-ad1a-4e97-af0c-89b86769101e\") " pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:40.169170 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.169175 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/365345c0-ad1a-4e97-af0c-89b86769101e-lib-modules\") pod \"perf-node-gather-daemonset-c5cbp\" (UID: \"365345c0-ad1a-4e97-af0c-89b86769101e\") " pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:40.169422 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.169198 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/365345c0-ad1a-4e97-af0c-89b86769101e-proc\") pod \"perf-node-gather-daemonset-c5cbp\" (UID: \"365345c0-ad1a-4e97-af0c-89b86769101e\") " pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:40.169422 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.169330 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5p9t\" (UniqueName: \"kubernetes.io/projected/365345c0-ad1a-4e97-af0c-89b86769101e-kube-api-access-p5p9t\") pod \"perf-node-gather-daemonset-c5cbp\" (UID: \"365345c0-ad1a-4e97-af0c-89b86769101e\") " pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:40.169422 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.169367 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/365345c0-ad1a-4e97-af0c-89b86769101e-sys\") pod \"perf-node-gather-daemonset-c5cbp\" (UID: \"365345c0-ad1a-4e97-af0c-89b86769101e\") " pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:40.269894 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.269801 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/365345c0-ad1a-4e97-af0c-89b86769101e-sys\") pod \"perf-node-gather-daemonset-c5cbp\" (UID: \"365345c0-ad1a-4e97-af0c-89b86769101e\") " pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:40.269894 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.269875 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/365345c0-ad1a-4e97-af0c-89b86769101e-podres\") pod \"perf-node-gather-daemonset-c5cbp\" (UID: \"365345c0-ad1a-4e97-af0c-89b86769101e\") " pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:40.270116 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.269899 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/365345c0-ad1a-4e97-af0c-89b86769101e-lib-modules\") pod \"perf-node-gather-daemonset-c5cbp\" (UID: \"365345c0-ad1a-4e97-af0c-89b86769101e\") " pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:40.270116 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.269920 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/365345c0-ad1a-4e97-af0c-89b86769101e-proc\") pod \"perf-node-gather-daemonset-c5cbp\" (UID: \"365345c0-ad1a-4e97-af0c-89b86769101e\") " pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:40.270116 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.269942 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/365345c0-ad1a-4e97-af0c-89b86769101e-sys\") pod \"perf-node-gather-daemonset-c5cbp\" (UID: \"365345c0-ad1a-4e97-af0c-89b86769101e\") " pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:40.270116 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.269973 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5p9t\" (UniqueName: \"kubernetes.io/projected/365345c0-ad1a-4e97-af0c-89b86769101e-kube-api-access-p5p9t\") pod \"perf-node-gather-daemonset-c5cbp\" (UID: \"365345c0-ad1a-4e97-af0c-89b86769101e\") " pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:40.270116 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.269979 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/365345c0-ad1a-4e97-af0c-89b86769101e-podres\") pod \"perf-node-gather-daemonset-c5cbp\" (UID: \"365345c0-ad1a-4e97-af0c-89b86769101e\") " pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:40.270116 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.270034 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/365345c0-ad1a-4e97-af0c-89b86769101e-proc\") pod \"perf-node-gather-daemonset-c5cbp\" (UID: \"365345c0-ad1a-4e97-af0c-89b86769101e\") " pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:40.270116 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.270081 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/365345c0-ad1a-4e97-af0c-89b86769101e-lib-modules\") pod \"perf-node-gather-daemonset-c5cbp\" (UID: \"365345c0-ad1a-4e97-af0c-89b86769101e\") " pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:40.280022 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.279983 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5p9t\" (UniqueName: \"kubernetes.io/projected/365345c0-ad1a-4e97-af0c-89b86769101e-kube-api-access-p5p9t\") pod \"perf-node-gather-daemonset-c5cbp\" (UID: \"365345c0-ad1a-4e97-af0c-89b86769101e\") " pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:40.298116 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.298090 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:40.439719 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.439696 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp"] Jul 24 14:25:40.441251 ip-10-0-137-60 kubenswrapper[2579]: W0724 14:25:40.441212 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod365345c0_ad1a_4e97_af0c_89b86769101e.slice/crio-234113ace4ec979d0cb306de8808febc1f987c3ff50ecd359c404d5b7e5ab142 WatchSource:0}: Error finding container 234113ace4ec979d0cb306de8808febc1f987c3ff50ecd359c404d5b7e5ab142: Status 404 returned error can't find the container with id 234113ace4ec979d0cb306de8808febc1f987c3ff50ecd359c404d5b7e5ab142 Jul 24 14:25:40.599596 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.599559 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" event={"ID":"365345c0-ad1a-4e97-af0c-89b86769101e","Type":"ContainerStarted","Data":"016a1bcb0f23dba894831fe02195ad21952db62c7b5007b8c750324ce5ab3c29"} Jul 24 14:25:40.599766 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.599620 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" event={"ID":"365345c0-ad1a-4e97-af0c-89b86769101e","Type":"ContainerStarted","Data":"234113ace4ec979d0cb306de8808febc1f987c3ff50ecd359c404d5b7e5ab142"} Jul 24 14:25:40.599766 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.599745 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:40.631721 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:40.631676 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" podStartSLOduration=1.6316603189999999 podStartE2EDuration="1.631660319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:25:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:40.624054171 +0000 UTC m=+3082.747436560" watchObservedRunningTime="2026-07-24 14:25:40.631660319 +0000 UTC m=+3082.755042716" Jul 24 14:25:42.313956 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:42.313914 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ck595_db90cec8-5114-4f7f-8966-79193d3cc584/dns/0.log" Jul 24 14:25:42.343524 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:42.343461 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ck595_db90cec8-5114-4f7f-8966-79193d3cc584/kube-rbac-proxy/0.log" Jul 24 14:25:42.426934 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:42.426903 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dkg7g_ea1c6809-b191-43e3-b132-2ceeddc0f058/dns-node-resolver/0.log" Jul 24 14:25:43.030484 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:43.030450 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6c46b45557-8b5kp_c1e0fc9a-b19b-4a76-aead-44c891fc755a/registry/0.log" Jul 24 14:25:43.147161 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:43.147135 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sv57j_b3048916-5eca-431c-84b6-8381da55db53/node-ca/0.log" Jul 24 14:25:44.128927 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:44.128902 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw_52323b36-8263-4bff-9712-b2b903cd4204/istio-proxy/0.log" Jul 24 14:25:44.283008 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:44.282972 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kxsk9_d685f64b-4b49-4930-b50a-b45df2488ba2/discovery/0.log" Jul 24 14:25:44.308035 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:44.307982 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85dc78bf9-5v5rs_ef5cbd04-8ca5-4930-82a3-a722b0ab36d3/kube-auth-proxy/0.log" Jul 24 14:25:44.409710 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:44.409628 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-l5w2m_0557c33f-f425-4707-90fc-085902eac820/istio-proxy/0.log" Jul 24 14:25:44.437443 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:44.437400 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-l5w2m_0557c33f-f425-4707-90fc-085902eac820/istio-proxy/1.log" Jul 24 14:25:45.427343 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:45.427317 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zz9l9_005b211d-95c2-4052-972d-61ac571c313b/serve-healthcheck-canary/0.log" Jul 24 14:25:46.114318 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:46.114287 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dj997_c3579589-b0f5-4d12-998d-892f6e682086/kube-rbac-proxy/0.log" Jul 24 14:25:46.152109 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:46.152080 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dj997_c3579589-b0f5-4d12-998d-892f6e682086/exporter/0.log" Jul 24 14:25:46.186677 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:46.186651 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dj997_c3579589-b0f5-4d12-998d-892f6e682086/extractor/0.log" Jul 24 14:25:46.615871 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:46.615845 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hh2p5/perf-node-gather-daemonset-c5cbp" Jul 24 14:25:49.015088 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:49.015048 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-h49g2_e088b46b-0f4e-4740-a42c-bcd1d04be8c8/manager/0.log" Jul 24 14:25:49.246224 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:49.246142 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4rfzs_4424e0d0-b6e6-44a0-82e9-a098570c212a/manager/0.log" Jul 24 14:25:49.461590 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:49.461563 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hfvdp_bd54ca4e-848b-4c2d-8be0-10214ff2148c/server/0.log" Jul 24 14:25:49.549466 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:49.549381 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-b4ln4_24b46733-ac33-433b-bf13-17bbbbdfc040/manager/0.log" Jul 24 14:25:49.748113 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:49.748078 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85fcbb566-fmjsf_ba47ec72-ddad-41a3-906d-61eae421e7ab/manager/0.log" Jul 24 14:25:49.766177 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:49.766155 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85fcbb566-fmjsf_ba47ec72-ddad-41a3-906d-61eae421e7ab/copy-manifests/0.log" Jul 24 14:25:50.456858 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:50.456830 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw_52323b36-8263-4bff-9712-b2b903cd4204/istio-proxy/0.log" Jul 24 14:25:50.595790 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:50.595758 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kxsk9_d685f64b-4b49-4930-b50a-b45df2488ba2/discovery/0.log" Jul 24 14:25:50.614344 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:50.614311 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85dc78bf9-5v5rs_ef5cbd04-8ca5-4930-82a3-a722b0ab36d3/kube-auth-proxy/0.log" Jul 24 14:25:50.675248 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:50.675217 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-l5w2m_0557c33f-f425-4707-90fc-085902eac820/istio-proxy/0.log" Jul 24 14:25:50.700512 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:50.700482 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-l5w2m_0557c33f-f425-4707-90fc-085902eac820/istio-proxy/1.log" Jul 24 14:25:52.176536 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:52.176509 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-6kjgw_52323b36-8263-4bff-9712-b2b903cd4204/istio-proxy/0.log" Jul 24 14:25:52.309687 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:52.309658 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kxsk9_d685f64b-4b49-4930-b50a-b45df2488ba2/discovery/0.log" Jul 24 14:25:52.322422 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:52.322400 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85dc78bf9-5v5rs_ef5cbd04-8ca5-4930-82a3-a722b0ab36d3/kube-auth-proxy/0.log" Jul 24 14:25:52.363757 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:52.363726 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-l5w2m_0557c33f-f425-4707-90fc-085902eac820/istio-proxy/0.log" Jul 24 14:25:52.388593 ip-10-0-137-60 kubenswrapper[2579]: I0724 14:25:52.388562 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-l5w2m_0557c33f-f425-4707-90fc-085902eac820/istio-proxy/1.log"